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ABSTRACT 

The Comprehensive School Mathematics Program (CSMP) 
is a program of CEMREL, Inc., one of the national educational 
laboratories, and is funded by the National Institute of Education, 
Its major purpose is the development of curriculum materials in 
mathematics for grades K-6. Beginning in September, 1973, CSMP began 
an extended pilot trial of its Elementary Pi^ogram. This report 
summarizes test data collected from second and third grade classes 
during 1975-76, the third year of the CSMP Extended Pilot Test. 
Included in this report is an introduction, descriptions of tests 
used, discussion of data, and Appendices. The appendices include data 
from the study and copies of most instruments. On standardized 
mathematics tests^ CSMP classes generally did as well as or better 
than non-CSMF classes. On tests relating to CSMP content, CSMP 
classes scored significantly higher than non-CSMP classes. Results 
were similar to data from previous studies. (RH) 
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Description uf Kvaluation Report Series 



The Comprehensive School Matnematics Program (CSMP) is a program of CEMREL, 
Inc., one of the national educational laboratories, and is funded by the National 
Institute of Education. Its major purpose is the development of curriculum 
materials for grades K-6. 

Beginning in September, 1973, CSMP began an extended pilot trial of its 
Elementary Program. The pilot trial is longitudinal in nature; students who 
began using CSMP materials in kindergarten or first grade in 1973-74, were able 
to usc' them in first and second grades respectively in 1974-75, and in second 
and third grader in 1975-76. Hence tl.'.e avljective "extended". 

The evaluation of the program in this extended pilot trial is intended to be 
reasonably comprehensive and to supply ifonnation desired by a wide variety of 
audiences. For that reason the reports in this series are reasonably non-technical 
and do not attempt to widely explore some of the related research istsues. The list 
of reports from the first tr^o years of the extended pilot trial is given on the 
next page. The most comprehensive of these are the following: 

1-A-l: Overview, Design aad Inscrumentation 

1- A-3: Final Summary Report, Year 1 

2- A-l: Final Suiimiary Report, Year 2 

and 3-;'.-l: Summary of Second and Third Grade Test Data Year 3 
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Key to Indexing 
Observations of CSMP First Grade Classes 



"2" refers simply to the number within a given year and type of data 

"C" refers to the type of data being reported 

A: Overview, summary and theoretical reports 
B: Student outcomes 
C: Non-test data 

"1" refers to the year of the Pilot Study according to the following: 
Kindergarten 
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Introduction 



This report summarizes test data collected from second and tlurd 
grade classes during 1975-76, the third year of the CSNIP Extended Pilot 
Test. This pilot test begcin in 1973-74 in kindergarten and first grades, 
with over 100 classes using CSND^ nuterials. In succeeding years, many 
of these classes continued with CSMl^ in progressively higher grade levels 
as the curriculum materials became available, and iiuiny new classes joined 
the pilot test. 

Much evaluation data was collected during the previous two years and 
is reported in tlic l-valuation Reports described on page iv. Tlie major 
enphasis in this 1975-76 study was at the second grade level, where classes 
were using a revised version of second grade materials (CSM^ Mathem;itics 
for the Upper Primaiy Grades Part I). Some data was also collected from 
third grade (CSMP Mathematics for the lij:)i:)er Primaiy Grades Part II). 

Three kinds of tests were used to investigate student learning: 

a) Standardized Mathematics Test 

b) CSi^P- Specific Tests 

■c) Mathematics .^\})plied to Novel Situations (MWS Test) 

A description of the content and results from each of these sets of 
tests is given in each of " the next three chapters respectively. 'Ihe GSMP- 
Specific Tests were givnsn only to CSMP classes, while the other two kinds 
of tests were given for conparison puiposes to both CSMP and Non-CSMP 
classes. Mien making such conparisons, scores from a general ability test 
or a reading test were used as covariates. Tlie use of such scores as 
covariates allowed scores frojn tests a) and c) at)Ove to be adjusted to take 
into account possible differences in ability iind taus eliminate, or at 
least reduce the plausibility of, the explanation that differences in 
perfoimances were a result only of differences in general ability. 

Second Grade rest ing 

In addition to classes from four school districts in the St. Louis 
area (Normandy, Fei'guson-I-Torissant, Ladue cmd St. Louis), test data was 
collected from several other school districts: New Hartford, N.Y.; Detroit, 
Mich.; Portland, Nki. ; Clai-ksville/Montgomeiy Coiinty, Tenn. ; Polk County, 
Ga. Haralson Co., Ga. ; cmd ttie Diocese of Marquette, Mich. Tlirough what 
were called *'Joint Reseai'ch Studies", each of these districts cooperated 
in collecting scme-or all of the test data described above (limited by 
sucli things as availability of conq^arison classes, testing time, etc.). 
The scliool districts, thi'ough the coordinators, were responsible for 
selecting conparison classes (wiiere availat)le), sch.eduling the testing, 
and training the testers. Wierever pcsible^ scores from regularly 
scheduled, district-wide tests w^ere used as the st<'mdardized maitheinatips 
test and as covariates. 'ITie CSMI^-Si)cci f ic Tests cind the MANS Test were 
developed by the CSMl^ LA'aluation staff. 

'Ihus a i-airly comj:)! icated design en^ergcd; in ])art a series studies 
from individual sites with somewhat different dnta collected from each, 
but w'ith enough cormionality that data could in S(^iik^ cases be comtvlned 
across sites. Uie of the ways in which this was done was by trrmsforming 
scores from the different covariates "^.o percentile ranks (though it is , 
recognized that percentile rank has soiiX^ undesirable properties, so that 
different nonaing ])i*ocedures must inevitat>ly lead to some discrepancies 
between different covariates). 
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The testing plan for second grade classes is sho\vn in Table 1, below. 
Site 01 was for classes in the St.. Louis area; otherwise sites are not 
identified beyond the ''site number'' given in Table 1, and used in sub- 
sequent presentations of results. 

Table 1 



Testing Plan, Second Grade 











Ninober o£ CSMP Clasaes/Noo^SMP Classes 


Number 


ot CSMP 


Classes, 




Ktf WUJ p o k l»a Qil 

Classes? 




Math Test 


Standardized 
Hach Test 


MA.NS 
Test A 


MANS 
Test B 


CSHP- 


Specific Tcscs 




Covariate 


TesC A 


TeaC B 


Individual^ 


01 


Yes 


Kuhlcoann Anderson 


CTBS^ 


15/13 


8/7 


7/6 


a 


7 




12 


Yes 


SAT-Re ad lag 


SAT^ 


6/6 


3/3 


3/3 


3 


3 


6 


13 


Yes 


CPT'Riiading 




6/6 


3/3 


3/3 


3 


3 


6 


25 


Yes 


Gates McCinicie 


CTBS^ 


6/6 


3/3 


3/3 


3 


3 


6 


32 


Yea 


MAT-Readtng5 


CTBS^ 


1/1 






1 






35^ 


Yes 


Lo r ge-Tho md ike 


CTBS^ 


2/2 


1/1 


l/I 


i 


1 


2 


4Z 


No 












3 


2 


5 


^3 


No 












6 


6 


12 



^Mutabec 6£ classes, from each ot which 4 students were cested. 

^CoiBprehenslve Test of Basic Skills; Macheaatlcs Cocoputation, and iMatheoatics Concepts and Applications. 
^Stanford Achieveoent T^sc; Matheoatics Computation and Mathecatics Concepts. 
^Cooperaclve Primary Test, Kacheiaatics 
^Metropolitan AchlevecufOt Test 

^Cowrl'ice data from this site not available In tlae to be Included in .analysis. 



lliird Grade Testing 

With the exception of two non-local third grade classes, from uiiom 
no reliable covariate data was available, the testing wls confined to 
three school districts in the St. Louis Metropolitan area. Altogether 
there were seven CSNF third grade classes locally and these classes 
were a part of the '*lead'* groi?) of classes who began using CSMP in first 
grade in 1973-74. 

night Non-CSNP classes were selected as comparison classes. 'ITiese 
were either the *'other** third grade class in the scliool or a class from 
an adjacent school. In the fall of 1975 both CSMP and Non-CSMP classes 
were administered the Kuh.lmann-.Anderson Test as a covariate. In the 
spring both groi^^s of classes were administered the mathematics subtests 
of the Conprehensive Test of Basic Skills; and each CSMP class was admin- 
istered one of two CSMP-Specific Tests. 

ITie chapters in this report have been kept fairly short and non- 
tecJinical. Much more detail can be found in the various appendices 
whioh make vp the bulk of the report. 
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Standardized Math Tests 



Second Grade 

In order to measure students* knowledge of basic crithiDetic skills, 
standardized tests were administered to CSMP and to No?5-CSMP classes. 
In the local St. Louis area (denoted '*site 0V\ hui mlc up of classes 
from four school districts), the Mathematics test of the Conprehensive 
Test of Basic Skills (CfBS) was administered by specially trained CEMREL 
testers. In Outer Ring sites, the usual policv v;as to use already 
existing data from whatever standardized test was routinely adminis- 
tered in the spring by tlie individual rchooi district. Although this 
made for good ecomony, it is inpossible to lunp these different sites 
togeth. T in one analysis, and the analysis was therefore done separately 
at each site. In the local area it was possible to also collect data at 
the item level. 



The general procedure for eacli site was to conpare cla*=^s means from 
CSMP classes with those from Non-CSMP classes, using scores from some 
tests of general ability or reading' to adjust for potential differences 
in ability between tliese two groips. For each site, the standardized 
math test used and the test used as a covariate for adjusting scores 
are given below. 



Site 01: 

Standardized Math Test: CTBS (raw score). 

At Level C (Grades 1.6 - 2.9), the mathematics section is 
divided into two separately timed tests: Mathematics Coiiputa- 
tion (28 items) and Mathematics Concepts and ^^plications 
(25 items). Tlie Conputation section consists of 10 addition, 
10 subtraction, and 8 multiplication problems. The Concepts 
and /\pplications section attenpts to measure skills in basic 
operations: numbers, numeration, measurement, and fractions. 
The problems are read aloud to the students, who select their 
answers from pictured, numerical, or printed responses. 

Covariate: KuhLnann-Mderson (raw score) 10/75 

Number of Classes: 15 CSMP; 13 Non-CSNP. 

Site 12: 

Standardized Math Test: Stanford Achievement Test (SAT) (percentile rank) 
Covariate: leading Test, SAT (percentile nmk) (5/76) 
Number of classes: 6 CSMI^; 6 Non-CSMi"^ 
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Site 13: 

Standardized Math Tv:;;f Cooperative Primar>' Test (CVY) (raw score) 
Covariate: Iteaiiin;^ T*'f>t, CPT (raw score) (5/76) 
Number of Classes: 6 CSM?; 6 Non-CSMP. 
Site 25: • ' 

Standardized Math lest: CTBS (raw score) 

Covariate: Gates McGinitie Reading Cominrehension (raw score) (5/76) 
Nunber of Classes: 6 CSMP; 6 Non-CSMP. 
Second Grade Results 

In order to conpare test performance of CSMP vs. Non-CSMP classes, 
an analysis of covariancc was perfonircd at each site, using class means 
as tlie unit of analysis. The results are sho\m in Table 2, below, includ- 
ing - for each site - means across classes before and after adjustment for 
the covariate and the p- value of the difference between CSMP and Non-CSMP 
means. (The p-valus is the probability of sucli observed differences occur- 
ring merely by cliance - a low p-valuc indicates that cluuice alone is im 
unlikely explanation.) 

Table 2 



Summary Data from 
Standardized Math Tests: Second Grade 
(First Entry: CSMP Classes,. Second Entiy: Non-CSMP Classes) 



Site (Uarober of USMP, 
Non~CSMF Classes) 


Covariate^ 


Computation 


Concepts and Applications 


Means 


Adjusted Means 


p value 


Means 


Adjusted Means 


p. value 


Site 01 (15,13) 
Comprehensive Tests of 
Basic Skills (Raw Scores) 


48.8 

46.8 


21.0 

18.6 


20.6 

19.1 


.11 


18.7 

17.8 


18.3 

18.2 


.86 


Site 25 (6,6) 
CTBS (Raw Scores) 


21.5 

20.7 


20.9 

18.2 


20.5 

18.6 


.11 


39.4 

35.3 


18.1 

17.5 


..AS 


Site 35 (2,1) 
CTBS (Raw Scores) 


54.8 

76.7 


24.5 

23.8 


too few classes 


22.4 

21.6 


too few clar'ses 


Site 12 (6,6) 
Stanford Achievement 
Teat (Percentile Ranks) 


57.7 

42.0 


54.9 

44.4 


47.0 

52.3 


.41 


53.3 

37.9 


43.5 

47.7 


.23 


Site 13 (6,6)2 
Cooperative Primary 
Tttst (Raw Scores) 


32.8 

35.4 




< 


« 


35,7 

36.3 


36.2 

35.7 


.74 
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^Covariate: Site 1: Raw score, Kuhlman Anderson Teat (10/75) 

Site 25: Raw score. Gates McGinitie Reading Comprehension (5/76) 
Site 35: Percentile Rank, Reading Test, Metropolitan Achievement Test (10/75) 
Site 12: Percentile Rank, Reading Test, Stanford Achievement Test (5/ *6) 
Site 13: Raw score, Reading Test, Cooperative Primary teat (5/76) 

^At Site 1.3, the Cooperative Primary Teat has only a total acore in nathcdiatics, which data has 

been put in the ''Concepts and Applications'* columns. 
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It can be seen that, while the differences were usuaily in favor 
of CSMP classes, in no case were tliese differences statistically sig- 
nificant. Tlie closest any differences came to significance (usually 
taken as <.05) was on the conputation section at Sites 01 and 25 
where the p- value was .11. /^pendices A (for Site 01) and E (other 
sites) provide information sunuiuirized according to class means. For 
each of the tests, the set of class means are plotted against the mean 
ability or reading score (transfomied to percentile rank) of eacli class. 
By inspecting these gra^^hs it is possible to deternune a.id conpare the 
extent to which various chisses do about as one would expect based on 
their mean ability scores. 

Where item analysis data was available (for the CTBS at Site 01) 
a conpilation was made of all items in whidi tlie difference in percent 
correct between CSMP students and Non-CSMP students was more than 10 
points. These items are sho\m below; in each case the difference was 
in favor of CSMP students. Conplete item analysis data 'for this test 
is given in /\ppendix A. 

13 

154 27 6 1.14 

Addition + 128 + x8 52 + ,63 26 + 21 = 

+ 27 



768 64 
Subtraction - 42 7 - 9 



— V 



Multiplication 5x2« Mx4= 2x3= 3x5= 3x2= 4x4= 8x5= 



Concepts & Applications 

U* FtA4 tb« clock chAt •hova tap thlrty~ft^» > 



o 



o 



o 



\3 

o 



15* find th« ouuraX chac shovs eoe-half of four* 



o 



2 

o 



3 

o 



. 4 

o 



13« flAd th« auMrU. thjc !• eot-half of »t» * 



o 



2 

o 



3 

o 



4 

o 



23* find th« QUBb«r lloe thaC thcwa forty-flva ehrc<* 



o 



o 
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Third Grade 

The Con^vrehensive Test of Basic Skills was administered to the 
seven local St. Louis area third grade classes and to eight Non-CSlvlP 
conparison classes. At Level 1 of the CTBS (Grades 2.5 - 4.9), the 
mathematics, section is divided into two separately timed tests: 
Mathematics Coniputation (48 items) and Mathematics Concepts and 
Applications (50 items) . 

Mathematics Conputation : llie 48 items in this test consist of 12 problems 
each in additim, subtraction, multiplication and division. 

Mathematics Concepts ^ Applications : There are 50 items in all. The 
25 '^concepts" it^ms purport to measure the student *s ability to con- 
vert concepts expressed in one numerical, verbal or graphic form to 
another form, and to conprehend numeiical concepts and their inter- 
relationships. The content categories for these items include 
number systems, measurement, set theory, geometry and numeration. The 
25 "applications" items purport to measure the student's ability to * 
select and carry out problem-solving operations. The content categories 
include set theory, algebra, measurement, and reasoning. 

ITiird Grade Results 

An Analysis of Covariance was performed using class mean scores as 
the unit of analysis. The mean raw score from the Kuhlmann -Anderson 
test, administered 10/75, was used as the covariate. The results are 
given in Table 3 below. 

Table 3 

Summary Data from tlie Conprehensive 
Tests of Basic Skills: lliird Grade, Local Classes 
(First Entry: CSMP Classes (n=7), Second l-ntry: Non-CSMP Classes (n=8)) 



Test 


Mean Scores 


Adjusted Means 


p value 


Covsrlate^ 


49.8 

53.2 






Addition Items 


10.0 

9.3 


10.2 

9.1 


.09 


Subtraction Items 


9.3 

8.9 


9.5 

8.6 


.18 


Hultlpllcstlon Items 


9.3 

9.:i 


9.5 

9.1 


.63 


Division Items 


7.7 

6.7 


8.1 

6.4 


.25 


Computation Test 
(total of above) 


36.3 

34.2 


37.2 

33.3 


.03 


Concspcs and Applications 


33. S 

31.5 


34.5 

30.4 


.02 


Total Math Score 


69.8 

65.3 


71.7 

63.3 


.01 



^Maan Raw Score, Xuhlmann Anderson Test, adalalfltered 10/^5. 
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Each of the four sub -parts of the Conputation test has been analyzed 
separately, though none was significant at the .05 level. However, the 
differences were significant for the Conputation test, for the Concepts 
and Applications test and for the total CTBS mathematics score ; and all 
differences, whether significant or not, favored CSNIP classes. 

Appendix C shows the distribution of class means. Also given in 
that appendix is item analysis data for all the math items of the CTBS. 
Given below are those items for which the difference in percent correct 
between CSNIP and Non-CSMP students was more than 10 points. (In each case 
the difference was in favor of CSMP students.) 



Addition: 



72+73 



32 
10 
+ 4A 



26+13+14 



Subtraction: 



1A9 
- 87 



25 

Multiplication: x A 



Division: 



3 IT" 2 IT" 5 |25 A [124 24 1 48 



Concepts & Applications: 



23 Which of tUw fotlowing makft tbtt 
number »tnt«nc« trtfa? 

3 X (2 X 4) - (3 X 2) X a 

(S> 4 ci) t; <D 8 <S) 24 

37 Wh 'C is Ihc miuini Rumbvr if. 
3.i>.? .11? 

(2> 1 <S>B 0 9 (S>10 



38 V.'h«l cncs in ihe boy to mak* thn 
numh:i srnlcnce truf? 

□ 7 - 4 8 

(DO (D6 <S) 12 (g) 19 

39 What ioc% in (he box to nuke (hit 
numbci u'nlenct true? 



5 4 - 10 - □ 
OO ID 1 (09 



<£) 10 
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CSMP-Specific Tests 

Second , Grade 

^Pwo group- administered tests, CSMP Test A and CSMP Test B, were 
developed to investigate student skills in working witli the second 
grade CSNF content. Tlie tests specifically excluded most of the 
traditional arithinetic skills and concepts of second grade since 
these were covered in the standardized mathematics tests described in 
the last chapter. Instead these tests focused on the special pedago- 
gical '^languages" of CSIvP: arrow diagrams, the Minicorfputer and string 
pictures . 

The forniat for tliese tests was that of a CSMP workbook, an eight 
page booklet printed in color on newsprint, llie printed directions 
were as similar as possible to those given in CSMP workbooks. The 
actual test pages were typical of the kinds of pages done by CSMP 
students on a regular basis; no new kinds of items were added nor were 
there novel applications or extensions of previous workbooks. 

Administration of these tests was on a sanpling basis; each CSMP 
class took either Test A or Test B, (The testing plan, across sites, 
is shown in Table 1, page 2.) These tests were not parallel tests; 
the sanpling was done for the purpose of reducing the amount of time 
required for testing in a given classroom. Specially trained testers 
administered the test in a fairly straightforward way. After the 
tester reviewed one or two of the pages with the class, the students 
sinply went ahead and worked right througli it on their own, just as 
they would an ordinary CSNP workbook. Students raised their hands when 
they had questiou.s, at which time the tester would explain the task 
individually to tlie student (though not, of course, how to do it). 
The test was essentially untimed. Nbst students took about 20 minutes 
to conplete it; but up to 35 minutes was allowed if necessary. 

For purposes of analysis, responses from pages dealing with similar 
areas of content were combined, though these pages miglit not be adjacent 
on the actual test. In this way the subtests named below were created. 
Also given below are sonx> summary data. 







Number 


Mean Percent 


Correlation with 


Reliability 






Items 


Correct 


Kuh Imann - Ande rson 


(KR20) 


Test A: 


y\rrow Diagrams 


12 


72 . 


.67 


.87 




Miniconputer 


10 


66 


.64 


.86 




Integers 


10 


50 


.39 


.76 


Test B: 


Arrow Diagrams 


17 


72 


.72 


.91 




Miniconputer 


10 


75 


.62 


.90 




String Pictures 


4 


65 


.51 


.63 




Computation 


/ 


69 


.74 


.80 



Except for the subtest ^Intepcrr?", the menn norcoTr-" rorroct ncross 
items for the various sul^tests was between 65 md 75 and the correlation 
with the Kuhlmann Anderson Trst was moderately high. 'Ihe Villi) reliability 
of the subtests was- also high with the exception of the very short (4 items) 
subtest ^'String Pictures". ' 
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An evaluation of the quality of student responses to this test is 
difficult for two reasons. 1-irst, because arrow diagrams and Miniconputers 
are not used in other niath curricula, tlie tests were adjninistered only to 
CSMP students, so that t:SMP - Non-CSNIP conparisons are not possible. 
Second, the test was not a masteiy test and there are no specified stan- 
dards or expectancies of the tasks and levels of success of whidi CSW 
students should be capable. Indeed,' because of the spiral nature of the 
CSMP curriculum, the concept of mastery is a much more difficult one to 
deal with. Summarizing statistics (sudi as mean percent correct) provide 
in themselves, veiy little useful infomiation to ^be reader in the absence 
of the actual tests and some knowledge of the cnrriculiun. 

Part of Appendix U presents information summarized across students. 
For each of the seven subtests of Tests A and B, reduced copies of the 
actual test pages are given, together with a) percent correct for each 
item, and b) summary statistics (means, frequency distr'Ibutions , correla- 
tions j across all students who took the test. Also given is similar infor- 
mation when students are grouped according to at)ility level, llias, it is 
possible to determine on varioas items and si±)tcsts, what kinds of differ- 
ences in perfonnance there were between higli and low ability students. 
Appendix D contains a graph for eadi subtest which shows the distribution 
of clasj* means as a function of class ability scores. 

In addition to the grou[j administered CSMP I'ests A and B, t\n individuilly 
administered test \<t\s constructed and administered to four representative 
students in eacli of 57 classes. 'Iliese classes were all outside the St. Louis 
area and the tests, requiring about 15-20 minutes per student, were adminis- 
tered by special testers trained by tlie CSNP coordinators in the various sites. 

The pur|:)Ose of this test was to investigate, in situations not 
amenable to paper and pencil, students* skill in dealing with various t^isks 
of the CSNP curriculum. Hence many of tlie test items were not typical of 
material found in tlie regular workbooks. Tliey were probably more oriented 
to teadier-led lessons than to workbooks; several being rather challenging, 
lliere is sim{:)ly no way to briefly summarize the results except to refer the 
reader to the actual test items :ind item statistics in .'\ppendix K. 

Third Grade 

Two group. adnun; stored tests, CSMP Test A and CSMP Test B were 
developed to investigate student skills in working with the third grade 
CSNP content, 'lliese ased the ''languages" of the curriculum unique to 
CSMP. 'Hie varioas test pages were not similar in all cases to previous- 
ly completed workbook pages; occasionally they were slight extensions 
of sud) pages. 

Diese tests were also adnun istered In- special testers. 1-ach of the 
seven third grade CSMP chusses in the jocnl St. Louis area took eillier 
Test A or Test R. 'llias, the item analysis data, sunnnarized in Ap]jendix 
F (which also reproduces the tost items), is based on at most four 
classes and no attemi.)t has been nuuie to further ;inalyze this data by 
student al^ility or by inGan class scores. 
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MANS Tests 

(Mathematics Applied to Novel Situations) 

Background 

Since the beginning of the CSMP Extended Pilot lest, considerable 
time and effort has been devoted in the evaluation effort to the devel- 
opment and use of tests of student outcomes which go beyond the 'T^asics** 
of the standardized tests yet do not rely on the specific ''languages'' of 
the CSMP curriculum. The rationale behind tJiis effort was as follows: 
CSMP is interested in developing a math curriculum f»;r elementary schools 
which would not only leave the ''graduates'' with the skills and concepts 
required in the traditional arithjnetic curriculum, but would go beyond 
that in giving the student a foundation of understanding for mathematics 
itself, 

Tlius it was necessary to devise test items vvhich would not only 
mtrasure a student's grasp of higher-order ideas in mathematics, but 
would also do so in sucli a way as to give no siperficial advantage 
to a student who had been exposed to mathematics in a particular way. 
Hence the items had to be at once (a) matliematical rather than merely 
aritlunetic, (b) ccntextually novel, and (c) free of language 
peculiar to CSMP or any other specific elementary math curriculum. 

During the first two years of the liPT, these items were groiped 
to form what were called "comparison tests." One can review the items 
dex'eloped and the results of their use in testing during those two 
years by consulting Evaluation Reports l-B-5 and 2-3-1. During the 
third year of the EPT, the name for these tests was changed from 
"conparison" to MANS in order to be more descriptively accurate. 

One bias should be pointed out here. Items were selected which 
were tliought to be related to basic mathemtical ideas behind tlie 
CSMP curriculum. Since the ideas are basic to mathematics, it might 
also be the case that otJier curricula would be aimed at giving students 
an understanding of the same ideas. Nevertheless, no attenpt was made 
to select the items conpletely at random with I'espect to tlie basic 
mathematical ideas involved. 

Description of MANS 

Tlie MANS test consists of ten subtests, divided into two booklets 
of five subtests each. The two booklets are not parallel in any sense, 
but have the sajne physical appearance and are administered in roughly 
the same way. Each booklet consists of eiglit 8^^x11 newsprint pages. 
Each subtest is contained on one or two pages. Because of testing time 
considerations any participating class of students either took booklet 
A or booklet B but not both. 

Specially trained testers administered the test in a fairly straight - 
foivard way. For each subtest the administrator gave directions orally and 
usually worked through tlie first item with the students. Each test was 
separately timed, so that all students began each new test together. 
Except for test A4 whicli was intentionally speeded, the time limits, 
based on pretesting, allowed most students to atten5:)t each item. Each of 
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the booklets usually took about 40 minutes each to administer. Students 
who conpleted the subtests quickly were told to check their work and then to 
draiv a picture on the fmnt cover of the booklet if they so desired. 

A sairple test item for eacli Mr\NS subtest is given below, together 
with a portion of tlae directions for that subtest. The full set of 
test items togetJier with more conplete directions is given in Appendix G. 

Booklet A 

Subtest 1: Sequences 'Ihe crucial directions were, "The numbers are 
in a certain order. . . figure out what the pattern is... put tlie riglit 
nunier in the box.'' 

Hxample: l'^, 13, 10, □, 4, 1 

Subtest 2: Equation Fluency ITie crucial directions were, write 
as many true nimiber sentences as you can, using only tliese signs and 
nunbers.'* 

Example: = + - x 1 2 5 

Subtest 3: Functions Tlie crucial directions were, "A teacher was 
playing a game with the class... a student gave her 2 (the first 
nuni)er of each pair), she did something to it and got 4 (the second 
number) ...figure out what it was that the teadier was doing to the 
nuirbers, and then put the right nuiifcer in the empty box." 



Example: 



10 



14 



10 



Subtest 4 : Number Line Estimation Ihe crucial directions were, ^^fliis 
is a funny looking number line isn't it? ...tliexc are a wliole lot of 
problems (13) and you won't have much time (2^^ min.) to do them. You 
should not try to calculate the exact answer; just decide quickly where 
the answer would probably go on the number line." 



29+29 

Example: ^ \ ^ 1 ^ 

" 10 50 100 500 1000 

Subtest 5: Coirputation Tlie crucial directions were, "...figure out 
what goes m the boxes to m;ike the number sentences true." 

Exanple: Q - 49 = 0 
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Booklet B 

Sijbtest 1: LabeHing Nunfcer Lines ITie crucial directions were, 
"...figure out what nuinber would go in the box on each nunt)er line/' 
(Note to reader: no student did both this, subtest and Sul^^test 4 of 
Booklet A in which the number line was not equal - intei^al . ) 

Hxajnj)le: ^ \- 1 f - I | -| ^ \ 

20 25 30 45 50 55 60 



Subtest 2: Nuinber Sentences Tlie crucial directions were, '"Write 
number sentences for 8... write as nvmy as you Cc'in tJiink of." 

Exanple: 8 = 

Subtest 3: Word Problems Tlie crucial directions were, "1-or each 
series of pictures there is a story. At the end of each stoiy there 
is a question you will be expected to ^mswer." 

Exanple: "First picture, 'Four cliildren each get the Siune 
allowance from their motJier. ' Second picture, 
'The four children put their allowances together.' 
Tliird picture, 'Tliey have altogether 12 dollars.' 
Question, 'How mucli did each child get?"' 

(Note to reader: while the above information was being read to the 
students, the student's test page contained the pictures below.) 




Su[:)test 4: Number Sentence Pictures 'llils was a "iivitdung" task; four 
equations on one side of the page and five clot, pictures on the other side 
The crucial directions were, "figure out which dot picture siiows feach) 
number sentence best... there are five pictures and only four numl.^er 
sentences ... make up a nurTl")er sentence that goes with the picture that's 
left over." 

1 • i ^•'^ ' • 1 

HxajT^^le: 5+3=8 l • • • I 

Subtest 5: Computat ion [Same directions as Su}>tost 5 in booklet A 
but different iteins.) 



MANS Results 
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At eacli of four participating sites all the CSMP classes and m 
equivalent nuiTil)er of Non-CSNP classes were tested. Tlie Non-CSNIP 
cl;is"si»^; 'Weu* '.^'Uutv'vl -o io pii^v uk* ;i v».t\Hij' rou-.^lih' t:<.iui vaU^nt 
{\\\th respcHt 1o >^rnri:^i ac.uierM*. ir iiu jiroup o\ L.V^l 

.classes. 'Hie sjune two booklets of the MVNS were given at eadi of the 
sites, with rouglily half the c:SNn^ mid half the Non-CSMF^ classes getting 
booklet A and half of cacli group booklet B. 'Hie testing plan in the 
various sites is shorn in Table 1, page 2. 

In order to compare test performance of CS>P vs. Non-CSMP classes 
an analysis of covariance on the class means was performed across sites. 
The mean scores at eadi site, adjusted for the covariate, are shown for 
CSNP and Non-CSMP classes in Table 4 below. Also sliown (riglit hand 
Culurms) are the mean adjusted scores confcined across sites and the 
resulting p- values for eacli of the MANS tests. 

Table 4 



Adjust.ed Mean Scores 
gy Site and. Mans Tests 
(First Entry: CSMP Classes, Second Entry: Non-CSMP Classes) 









Ittf^' 














01 , 


12 , 


13 






25 


Combined 




HANS A 




















Covariate 


3B. 


5 

50.9 


50.4 

45. 1 


62.9 

68. 


2 


49. 


42.8 


56.1 

54,0 


1 


f 




6 


2.4 

2.7 


2.8 

2. 


3 


3. 


8 

2.9 


2.9 

2.5 


,09 


2. SquncLon 
Fluency 


3. 


3.4 


2.9 

5.2 


3.9 

3. 


1 


4. 


7 

4.0 


3.7 

3.4 


,43 


3. Funcdons 


1.8 

1.2 


1.8 

1.4 


2.0 

2. 


T 


2. 


3 

1.7 


2.0 

1.6 


,09 


4. Nutabcr Line 
Ksdoac Lon 


4. 


3 

3.8 


3.3 

4.0 


4.2 

4. 


•» 


5. 


7 

U,b 


4.4 

4.1 


,23 


S. Coopucatlon I 


b. 


4 

5.0 


7.2 

5.7 


7.2 

5. 


0 


7 


4 

6.8 


7.0 

5.6 


,01 


TOTAL 


18. 


7 

15.6 


17 6 

17.2 


20.1 

16. 


3 


23 


9 

19.9 


20.0 

17.4 


,04 


KAtfS B 




















Covariate 


59 


6 

54.5 


64.7 

41.3 


61.3 

7'* 


2 


40 


.9 

41.8 


56.7 

53.3 




1. Nuxcbcr Line 
Ubclllng 




4 

4.0 


4.7 

6.3 


4.7 

4 


1 


6 


.2 

6.2 


5.0 

5.1 


,88 


2. Sentences 
About 8 


3 


7 

2.9 


3.1 

3.9 


3.4 

I 


.9 


4 


.8 

5.0 


3.7 

1.4 


,19 


3. Word Problema 


4 


.0 

3.8 


4.2 

4.0 


4.5 

3 


.7 




.7 

4.5 


4.3 

4.0 


,08 


4. Nuaaber Sentence 
Pictures 


4 


.8 

4.2 


5.3 

4.7 


5.6 

3 


.7 


6 


.5 

J. 6 


5.6 

4.5 


.01 


S. Cooiputatlon II 


6 


.0 

4.8 


6.7 

5.9 


6.7 

3 


.5 


8 


.1 

7.3 


6.9 

5.4 


,01 


TOTAL 


22 


.9 

19.7 


23-9 

25.0 


24.9 

16 


.8 


30 


.3 

28.6 


25.5 

22,5 


,02 



^Htnibar of claaaaa taking HANS A in sites 01, 12, 13 and 25 was 15. 6, 6 and 6 respi^ctively : 
AC aach aita approxlMtely half were CSMP claaseM and half were Non-CSMP claaaaa. 
* For HMS B« tha nui*ar of claases watt 13. 6, 6 anil 6 reapectivaly and they vera different 
Q ' claaaaa than thoae who cook MANS A. 
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A review of Table 4 reveals that there are statistically significant 
differences in favor of CSMP classes (p<.05) for the total score on MANS A 
and on M/\NS B and for tliree of the Individual sul)tests: Confutation 1, 
Nunit>er Sentence Pictures and -Consultation M. ()n tliree other subtests — 
Sequences, Functions and Word Problems ~ the differences were nearly signifi- 
cant (p<.10). IVhen examining tlie scores for tlie combined sites, it can be 
seen tliat all differences, whether significcuit or not, are in favor of the 
CS^P classes • This pattern is replicated at most sites, altliough four of 
the twelve scores at site 12 are in favor of Non-CSMP classes (thougii not 
significant). 

Test performance as a function of ability level was examined for 
each subtest by grouping the students into quintiles according to 
student ability as defined by covariate score. Por exaiiple, a mean 
score was computed based on all CSMP students whose covariate score 
corresponded to a percentile rank less than 20 (lowest quintile). 
Similar mean scores were computed based on CSMP students with covariate 
percentile rank from 20 to 40 (second quintile) , 40 to 60 (third 
quintile), 60 to 80 (fourth quintile) and 80 or more (highest quintile). 
Then the same thing was done for Non-CS^D^ students, 'flie graphs in Figure 
1, below, allow visual comparisons of CSMP and Ncn-CSMI^ students at eacli 
quintile, and one can judge the degree to whicli overall higher scores by 
CSMP students is caused by students at a particular ability level. 

It can be seen in Figure 1 that on each subtest the CS>P mean for 
the various quintiles is almost always larger (higher on the graph) 
than the corresponding Non-CSMP metin. In otlier words CSMP students arc 
doing better than Non-CSMP students; not just overall, but at each ability 
levol. it is at tlie fourtii quintile (percentile rank 60-80) that the 
largest rjid most consistent differences occur. 

^\ppendix L also presents an item iuialysis of the M-\NS tests, with 
percent correct for eacli item across CSMl^ tmd Non-CSMP students. Included 
with the item analysis are graphs showing the distribution of class means 
for eacli M'\NS test according to class ability (covariate) scores, and 
various other summar>* statistics. 



o 
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Figure 1 

CSMP ( — ) and Non-CSMP (■ • -) Means by Quintile 
(The vertical axis Is for raw scote, with one unit equal to one half of a raw scove 
point, except for the two total scores where one unit equals two raw scoJ/poinJsO 
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Discussion 

After a brief treatment of the data obtained from the CSMP-Specific 
tests, the bulk of this discussion will focas on the data resulting from 
CSNP " Non-CSMP comparisons. 

Although a great deal of infonmition is available from the various 
CSNIP-Specific Tests, the spiral nature of the curriculum and consequent 
. lack of mastery levels at any given point in time make it difficult to 
assess the adequacy of student perfonriOi' in this area. It does appear 
for the most part that student.^ were moderately successful in doing the 
workbook- level pages covered in these tests. (On the second grade CSMP- 
Specific Tests A and B for example, with the exception of the subtest 
'*Integers'^ the proportion of correct answers was between 2/3 and 3/4 
for each subtest. In fact, of the total of 60 test items, the percent 
correct was between 70 and 90 for 62% of them.) However, it is clear 
that proficienc)^ with Minicomputers and arrow diagraji.s are not particularly 
important objectives per se. Wliat is inportant to know is how well a 
curriculum with these pedagogical devices succeeds in having childrOia 
learn' mathematical information (sucli as niuiierical concepts tind coiT^:)utational 
skills) and learn to apply tliis infomation usefully. 

With regard to comparisons between CSMP and Non-CSMP students the 
pattern of results is very similar to the pattein in previo^js years of 
the Extended Pilot Test, with one veiy important exception: these results 
are from a much broader base of classes in terms of geographic location. 
In particular, these results have been extended to sites outside the 
St. Louis area and are thus free from tlie certain non-generalizable 
features sucli as teaclier training by CSM^ personnel , knowledge of being 
a ''horserace'', frequent observations from visitors, etc. 

Briefly one can summarize the results as follows: 

a) On standardized mathematics tests, t:SMP classes generally did as well 
as or better than Non-CSNP classes. At third grade the differences 
were significant on botJi tests of the CTBS. At second grade the 
results were more equivocal; not readying significance at any of the 
four sites, but usually in favor of CSNil\ especially on the tests of 
conputational skills. Similar results have occurred in previous 
years; small differences in favor of CSMP students which sometimes 
readied significance and sometimes did not. 

b) On the MANS tests, CSMP classes scored significantly higher than 
Non-CSMP classes on both total MANS scores and on three of the sub- 
tests, and were close to significantly higher (jx.lO) on three 
other subtests. Again tliis pattern is rather similar to results 
from previous years: consistently higher scores by CSMP classes 

in all subtest's with significance reached on certain subtests and 
on tlie overall totals. 

ihe value one attributes to the superior performance on the M^\NS 
tests is of course dependent on tlie inportance one gives to tlie mathe- 
matical abilities and thinking processes required by them. It is true 
that these tests are biased to the extent that tliey were developed with 
the CSMP curriculum and pedagogv* in mind and that a different set of 
tests might have been developed with entirely different i-esults. Never- 
theless the actual tasks of the MANS tests were usually quite different 
from those typically found in either the CSMI^ curriculum or in most 
elementary school math texts; they were novel to both groups and used 
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none of the special icinguages of i:s.\n\ Such tnuisfer effects arc rela- 
tively unusual in curriculum evalmtion. 

It is also tnje that certain niathenutical content embedded in these 
tasks may favor the CSMl' classet^. if one looks at the percentage 
correct for CSMP and Non-CSMP classes on an item-by-item basis, ojie 
finds that several of the items on which there are substantial differences 
in favor of CSMI"^ have to do with: a) multiplying, b) using fractions or 
c) working with *'larger*^ numbers (in the hundreds). CSMP does stress these 
ideas somewhat more than most math prograjns. llius, at least some of the 
differences between CSMP and Non-CSMP classes on the M^\NS tesFmay be due 
to a difference in the mathematical content covered by the respective 
curricula. However, whether one views it as knowing more content or as 
being better able to use certain processes, the gains for ':SMP seem to 
have been aclneved without any C0iT^:)ensating loss in other a/eas (at at 
least none that showed up on the standardized measures). 

Given for t)\e moment tliat the favorable outcomes of tlie CSMP - Non-CSMP 
comparisons are accepted, it is necessary to consider alternative explanations 
for these outcones before concluding that the CSMP curriculum made the 
difference. Probably tlie most plausible of these is that CS^P teacliers as a 
whole may have been a better (or more enthusiastic) groi?) of teachers than 
\<ere Non-CSMP teadiers. 'Iliey may have been selected (or volunteered) because 
o£ certain favorable attributes, or may have developed same as a result of 
this selection (or volunteering). 'Hiere is no way to adequately test this 
possibility after the fact; researcli has sho\>/n that objective criteria of 
teacher attributes generally do not bear a consistently strong relationship 
to student acliievement. llie use of a design whereby volunteer teacliers are 
randoialy assigned to CSMl^ and to Non-CSMP classes was not practical in this 
study, nor is it generally practical in educational settings. 

(Now it is true that the e>q)lanation of teacher differences is most 
plausible during the first year of a stud>-, when teadiers may well have 
volunteered; in subsequent years teacliers' to sokic degree "inherit" the 
CSMP class from tlie previoa; year - though some selection still may take 
place. It is also true that a previous study found that teachers in their 
second year of CSNP did at least as good a job of teacliing CSflP as they 
did in their first year, as measured by student achievement (see Evaluation 
Report 2-B-2). Thus any decrease in teaclier enthusiasm, at least from 
first to second grade, did not appear to affect student achievement, lliese 
considerations tend to reduce, but not eiindnate, the hypothesis that teacher- 
differences are responsible for CSMI^ - Ncn-CSMF differences.) 

/uiiong CSMl' cUisses there is considerable variation in: a) the iimount and 
kind of training received by teachers, b) the amount of time per day devoted 
to mathematics instmction ivnd c) the iimount of sui:)plen)entaiy materials, su± 
as drill sheets, which are assigned to students. It is not clear how much 
corresponding variation in adiievement, if any, results from eadi of these 
factors. From i\n experimental point of view, they may be regarded as rather 
natural variations in the implementation of CSMP in the "real world". And 
there are similar variations in Non-CSMl^ classes. (For exauple, most teachers, 
no matter what curriculum they are using, routinely assign sui:)plemontary 
material of some kind; some more th:m others.) It is for this reason that one 
uses classes as the unit of iinalysis; a more consei-vative but realistic n)ethod 
than using individual students. 
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Finally one should note that considerable credibility is added to 
the results by their consistency. The results have been fairly consis- 
tent from year-to-year, across different levels of student and class 
ability, and in different educational settings and school districts. 
Bearing in mind the potential effects of teacher selection, these results 
offer considerable promise for the CSMl^ curriculum. 



Introduction to Appendices 



Two methods of data presentation have been used in various appendices. 
One method involves aggregating across students to get percentages correct 
on individual test items; the other involves the calculation of mean scores 
for each class and then using the set of class mean scores to carry out 
tests of significance. These are described in more detail below. 

First, the actual test items making up a particular test are given. 
Beside these items are given percentage correct for all CSMP students 
and for all Non-CSMP students (where the test was also given to Non-CSMP 
students). Occasionally, where covariate scores (reading or general 
ability scores) were available, this analysis was also carried out by 
level of student ability. Hiis was done by transfoimng covariate scores 
to percentile rank and doing separate analyses for the set of students 
mth percentile rank less than 20 (lowest quintile), the set of students 
with percentile 'rank 20-39 (second quintile), and so on to the set of 
students with percentile rank of 80 or more (fifth or highest quintile). 

Second, when cl;iss means are being analyzed for some particular 
test, ;;iean score is calculated for each class on that test and on the 
coVc-;?*?ar..e test. Tlien, by transforming the covariates to percentile 
ranks, each class can be represented by a dot, as illustrated below. 




V Covariate 

^ Score 

The vertical axis is for mean class test score on whatever test is 
being analyzed; the further towards the top of the graph - the higher 
the class score on that test. The horizontal axis is for covariate 
score, or mean class score on whatever covariate was used; the further 
to the right - tlie higlier the general ability of the class • Based on 
the set of class means thus graphed, the regression line has been 
drawn. This line is the best linear prediction of mean class test score 
that can be made from knowing the ability level of the class • Note in 
the exairple that classes A and B fall well below the regression line, 
or are scoring well helm what would be predicted for them from knowing 
the ability level of the class, while classes C and D fall well above 
the regression line. Note also that, although class A had a slightly 



EKLC 



2b 



higher mean score on the test than did class 1), class I) did nrudi better 
given the relative ability scores of the two classes. IVlien the class 
means generally fall close to the regression line, test scores are 
well predicted by the covariate; when they are more dispersed from ,this 
line, tlie covariate is a less effective predictor. ' 

For tests in which CSMP - Non-CSMP conf)arisons were made, summary 
statistical data has been shown in some unused comer of the graph. The 
method of conparison was the Analysis of Covariance, using class Ji)ean 
as the unit of analysis. Essentially this method assumes that at legist 
part of the test score differences among classes is due to differences 
in general ability levels of the class (as measured by the covariate) 
and test scores are adjusted to take into account such differences • 
The p-value shown at the bottom of these tables is essentially the 
probability that, after taking into account differences in class ability 
levels (covariate), any observed differences between CSMP and Non-CS^P 
class scores are random '^errors" from the true state of affairs - which 
is "no differences between CSMP and Non-CSMP." A low p-value indicates 
that this is an unlikely explanation of the true state of affairs; that 
in fact there probably are real differences between CSNF and Non-CSMP 
classes • Traditionally a p-value less than .05 is interpreted as 
"significant". 

It should also be noted that all analyses, whether aggregated 
across students or across classes, are based on students who were also 
present for tjie administration of the covariate test, utiere such was 
given (i.e* where comparisons were made). This usually elininated very 
few students from the analysis and tended to do so in about equal pro- 
portions from CSMP and from Non-CSMP classes. 

For the CSMP-Specific Tests (^pendices U, Hi and F) considerable 
use of color was made on the actual tests. The reproductions given 
here are in black and white, with various "tliicknesses" of lines taking 
the place of different color. The results are fairly easily understood, 
though not an aesthetic success. 

At the end of most Appendices, a brief Commentary is given, in 
whicli various points of interest (at least to the authors) are noted. 
To aid the reader, subsequent pages are labelled with both the page 
number and the letter for the ^pendix (i.e. " 115" means page 115 
of the report and part of Appendix F) . F 



/Vppendix A 

Conprehensive Test of Basic vSkills, Second Grade, Site 01 

lliere are two iiiathematics tests in the Conprehensive Tests of Basic 
Skills: Conputation, and Concepts and Applications. Fof~the purpose of 
this analysis, the Conputation Test has been subdivided into three 
sections: addition iteirLS, subtraction items and multiplication items. 
These sets of items are also separate sections in the test but are 
ordinarily combined to form a gross Conputation score. 

For eacli subtest, two kinds of information are given. First, a 
reduced version of the test items is given (usually on the left hand 
page) together with percent correct for CSMP students and for Non-CSM^ 
students. An asterisk indicates that the difference in percent correct 
was greater than 10. Ml such differences favored CSMP students. 

Second (usually on the rigiit hand page), a graph of class msaiis is 
given iN^iich corresponds to the set of test items on the facing page. 
CSNF classes are shown by an x and Non-CSMP classes by a dot (#) . Also 
shown on eacli graph is a statistical summary of tlie class mean data, 
including the test of significance for differences between CS^'^P and 
Non-CSMP classes. 

Included in the analysis were 272 CSW students with a mean raw 
score on the Kulilmann-/\nderson Test (administered the previous fall) of 
48.2 and 251 Non-CSMP students with a mean score of 46.3. 

Tlie information is presented in tlie following order: 

addition items 
subtraction items 
multiplication items 

Conputation test (sum of the above sets of items) 
Concepts and Applications Test 



22 

A 



CTBS Item Analysis, Second Grade, Site 01 



Subtfcst 1: Addition (10 items) 



Percent Correct Percent Correct 





CSMP 


Non-CSMP 




CSMP 


Non-CSMP 


3 
8 
0 

+_6_ 


87 


89 


13 

6 
52 

4 27 


64 


49 4 


19 

+ 6 


83 


76 


40 + 30 - 


91 


87 


154 

+128 


80 


64 ^ 


7 + 34-2 = 


88 


91 


27 

+18 


68 


57 ^ 


11 4- 7 = 


90 


88 


1.14 
+ .63 


41 


28 :^ 


26 + 21 - 


83 


69 ^ 
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Site 01 



CTBS 
Addition Items 
(Raw Score) 



CTBS (additi6ir items) versus Covariate (Kuhlmann Anderson) 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 

ririT 



So 






Site 01 




CSMP 


Non-CSMP 


Number of Classes 


15 


13 


Covariate Mean' 


46.8 


46.8 


Addition Items: 
(Adjusted Mean) 


7.7 


7.1 


Probability level 
of differences 


.03 



^Display on graph used percentile ranks 
statistical analysis used rav scores. 



Covariate 
(Percentile Rank) 



CTBS Item Analysis, Second Grade, Site 01 
Subtest 2: Subtraction (10 items) 



Percent Correct Percent Correct 





CSMP 


Non-CSMP 




• 


CSMP 


Non-CSMP 


- 6 


79 


78 




64 
- 9 


68 


57 


545 
- 33 


71 


64 


65 


- 42 = 


80 


75 


16 
- 8 


72 


73 


24 


- 8 = 


71 


67 


768 


74 


60 


13 


- 8 = 


66 


69 


55 
-22 


65 


61 


78 


- 43 - 


64 


54 



Site 01 

CTBS (subtraction items) versus Covariate (KuhliMnn Anderson) 
Sei:ond Grade ClaHs H^ans: CSMP Classes (x) and Non-CSMP Classes (•) 




26 
A 



CTBS Item Analysis, Second Grade, Site 01 



Subtest 3: Multiplication (8 items) 

Percent Correct 



5x2- 
1x4-- 
2x3- 
3 X 5 « 



Percent Correct 



CSMP 


Non- 


•CSMP 








CSMP 


Non-CSMP 


82 


68 




3 


X 


4 - 


81 


64 


83 


67 




5 


X 


9 = 


64 


58 


83 


64 




4 


X 


4 = 


71 


52 


81 


65 




8 


X 


5 = 

i 


71 


55 



ERIC 
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Site 01 

CTBS (multiplication itetr.s) versus Covariate (Kuhlinann Anderson) 
Second Grade Class Means: CSMP Classes (x) and Non-CS}IP Classes (•) 



CTBS 

Multiplication Items 
(Raw Score) 




^tERJC 30 



Covariate 
(Percentile Rank) 



CTBS, Second Grade, Site 01 

The graph on tlie next page is for the Confutation Test \vliich is 
conposed of the three previous sets of items for addition, subtraction 
and multiplication. 



< Site 01 

CTBS Coraputation versus Covariate (Kuhlniann Anderson) 
Second Grade Class Means: CS}IP Classes (x) and Non-CSMP Classes (») 
(Note; Computation Test is composed of the- addition, 
subtraction and multiplication items.) 
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CTBG Item Analysis, Second Grade, Site 01 



Subtest 4; Concepts and Applications (25 items) 



Percent Correct 



I. tLad • box thar. h«» • vj of thowliii tMn*^* 



a > 2 
O 



U > 2 
O 



11 ♦ \ 

o 



10 2 

o 



2* find cKtt box v1i<t« the ouabat word C*!!** ^o^ r^An y bo«t« 



CSMP 
84 



78 



Coux 

o • 



tvo 

o 



O 



3» find the DU»»!*J tJu*t shows two hundrgd y^ rh r«'» ceps, 
and mix ono. 



23C 
O 



263 

o 



326 

o 



o 



3<2 
O 



4. find thtt box lUac show* couaelog ^7 flvrj, ^ixrttag se 23. 

15, 30, 35. AO 25. 28. 31. 34 25. 27, 2^. ' 25. 35. 45. 55 
0000 
3. find tta* box th<c shows counclog by tvo<. tttitlng st 77. 



77, 73. 69. 6f. 
O 



77. «. 85. 89 

o 



77. 78. 75^. ^-a 

o 



77. 79. 81. 83 

o 



ft. fiad ths bo* trt*t show« food that U iol:! b/ t h« qtAmrt . 



«E1 

o 



o 



o 



7. rtack ths circls uadsr th« <aount of H>on «> thcrsm 1a Ch« 
plctur*. 






AU 

O 



36e 
O 



iot I 374 
O O 



t. find Ch« boK th*c shovs cha ma^ aajtjic of noney • 



o 




82 



78 



70 



76 



79 



89 



Non-CSMP 
87 



75 



82 



80 



60 



75 



73 



87 



4 » 



CTBS Item Analysis, Second Grade, Site 01 
itest A: Concepts, anjj Applications (cont.) Percent Correct 



ERIC 



f* D«t14 Voufht • klCa tkaC cotC olaaCaan eanca. Ba gav« 
C>M ftora Clark a qu^rtar. Plnd eha box chal ah«jwa 
bcw# >uch channa 0 « vtd got back . 



10. floadMj la eha •^cxm^ day of Che week. ftnd this box that 

•hovv vhaC da/ oS cha weak Thursday la. H^rk your anrwar. 



4 

o 



s 

o 



6 

o 



7 

o 



!!• *lod Cha dock cnaC ahova ftgceiro aCnurea nftmr sevfo 



o 



© 0 ! 0 



o o 

12* ftad the clock thutC ahova Can chlrcy-flve . 



o 



o 



o 



O 



o 



13. find the oiA«ral thJt U ona-^hait of al« . 



o 



2 

o 



3 

o 



o 



Flad thm box chat ahows ChaC Chree-fourCha of tH' 
ycCanlle la djck. 



EC 



o 



o 



tS. Fiad the nuaar^ tii*t ahova oo^*half of fot-r . 



1 

o 



2 

o 



3 

o 



14. riad Cba QMabar accitanca that la tnia . 



4 

o 



O 



It 



I • • 

o 



14 



6 * S < 

o 



17* FIa4 the aunbar a«acenca tbac i» Crua. 



• • S ♦ C 

o 



to • 

o 



J • t 



ft * « 



J ♦ V 

o 



o 



li« HUm acor«d four polnta for bU taaa. Hla brocyiar 
■cored evo points. UhJ-ch box shows how co fLnd the 
■^^ag of joints cha C_Mp bova scored cotjechor/ • 



C « ) 

O 



4 - t 

o 



19. 



4 » S 



1*1 



Oom day rrank cau^t alx flahJ Tbe next dey he caught 
•O fi«h. rind Che box chet ahows how aaoy ft?*h _LrAnk 
cauihC all toReeh^r In the cwo dayt . 



• 




\4 


•| 


















c 




0 


0 


0 


Jm lavltad elx boya and eight Ktrla to a 


party. Find 


Che box thaC 


ehoua h.rw nalnv children h#» <nvlced wU 














11 




u 


1> 






o 




0 


0 







O 



CSMP 


Non-CSMP 


39 


51 


AO 


45 


72 


73 


64 




82 


67 ¥• 


90 


n ^ 
96 


31 


67 


86 


86 


82 


77 


78 


76 

1 


84 


77 






89 


88 
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Subtest 4: Concepts and Applications (cont.) 



21* Joan Km thraa craycna, 3u« hat mix cra/ona, and Judjr 

aavan crayons. find Ihe box ChaC ihosn hav lynv 
crayon> tha i.l rln h>ve »l l to^yghtf . 



Aidy 7 



O 



o 



It 

o 



17 

o 



22* Tharc arc Cvcncy-alj^t children In a clui. On Friday^ 
fl^ chlL4rao vera jbs-snC. Which box shows hov to find 
Ch« o*«b«r ot chtldrro th^c war* ac school ou Friday? 



o 



o 



24 ♦ I 

o 



o 



23* fmi tb« ausiiar Xloc t*' ♦howa forty-ftv^ plua Ch r4ge. 



24* lAtoy gati up «c »r>/rn u^clocl. Ha ihould ba at achoo) 

two hours later. Flrd the box chat shows vh-iC tie*; L »ro^ 
ahouX4 at ichool. 



roo 

O 



brO«5 

O 



1:00 

o 



o 



25* fiad tha box vhara nt^a^ ? i» tha Klsalng aiAbcr 
7 



O 



O 



ij - a - • 

O 



i« - o 
O 



Site 01 

Percent Correct 


CSMP 


Non-CSMP 


81 


8A 


61 


57 


71 


5A 


61 


52 


53 


53 



ERIC 



Site 01 

CTBS Concepts versus Covariate (Kuhlmann Anderson) 
Second Grade Class Means; CSMP Classes (x) and Non-CSMP Classes (•) 
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Commentary 

Overall, these results were similar to those obtained in a similar study 
last year (see Evaluation Report 2-B-l). CSMP and Non-CSMP students got approxi- 
mately the same score on the Computation Test this year as did CSMP and Non-CSMP 
second graders respectively last year (after differences in ability were taken 
into account). The differential in favor of CSMP, about a point and a half each 
year, was significant last year; not quite significant this year. This may be 
because of the less restrictive (and less powerful) experimental design in the 
present study, where Non-CSMP classes were not generally selected from the same 
school as CSMP classes, which eliminated the possibility of a two-way Analysis 
of Covariance, 

Of the three sets of 'items making up the Computation Test, this yearns 
CSMP students d*id relatively better in addition and worse in subtraction than 
similar students last year. This yearns Non-CSMP students did relatively better 
in addition and worse in multiplication than similar students last year. 

On the Concepts and Applications Test, the differential between CSMP and 
Non-CSMP students was virtually nil this year; almost a full point last year. 
The difference is due to this year's Non-CSMP group outperforming last year's. 

At the item level, the pattern of responses was also similar to last year's. 
The set of asterisked items in this Appendix, which most differentiated the perfor- 
mance of CSMP and Non-CSMP students, is almost identical to the set of items with 
the largest differential last year. It is hard to see any particular pattern from 
this set of items, but the results seem to be consistent. 

It is worth noting that the regression model did not fit the set of CSMP class 
means particularly well for this data; in fact poorer than for any others in 
this report. A glance at the graphs in this Appendix will reveal that the CSMP 
class means tend to be curvilinear rather than linear, i.e. 
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/^pendix B 



Other Standardized Test Data, Second Grade 

The previous Aj^pendix, Appendix A, provided item and summary data 
from the math tests of the Conprehensive Tests of Basic Skills adminis- 
tered in Site 01. Appendix B provides data from tlie administration of 
standardized tests in mathematics at three other sites. In eacli case 
class means for CSNIP and Non-CSMP classes are plotted on a graph against 
the class mean covariate scores. Summai)^ statistical information, 
including adjusted means and the probability level of the differences, 
is presented in one comer of the graph. 

The graphs which are given, and the standardized math tests used 
are the following: 

Site 12: Stanford Acliievement Test, Conputation Test 
Stanford Achievement Test, Concepts Test 

Site 25: Comprehensive Tests of Basic Skills, Conputation Test 
Conprehensive Tests of Basic Skills, Concepts and ^ 
/^plications Test 

Site 13: Cooperative Primary Tests, Mathematics Test 



SAT 
Computation 
(Percentile Rank) 



6? 



55 



3? 



Site 12 

SAT Computation versus Covariate (SAT Reading) 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 



X 



Analysis of Class Means 






Site 12 




CSMP 


Non-CSHP 


Number of Classes 


6 


6 


Covariate Mean 


57.7 


42.0 


SAT CoiTiputation 
(Adjusted Mean) 


47,0 


52.3 


Probability level 
of differences 


.41 



15 



30 



to 



6r 



Covariate 



70 (Percentile Rank) 



Site 12 

♦ 

SAT Concepts versus Covariate (SAT Reading) 
Second Grade, Class Means: CSMP Classes (x) and Non-CSMP Classes (i) 



SAT I 
Concepts j 
(Percentile Rank) j 




Covariate 
(Percentile Rank) 



Site 2fi 

CTBS Coiputation versus Covariate (Gates McGinirie) 
Second Grade Class Means: CSMP Classes (>:) and Non-CSMP Classes (•) 




Analysis of Class Means. 





Site 25 




CSMP 


1 

Non-CSMP 


Number of Classes 


6 


6 


Covariate Mean 


21.5 


20.7 


CTBS Conputaiion 
(Adjusted Mean) 


20.5 


16.6 


Probability level 
of differences 


.11 



IZ — W 



If 



25 



Covariate 
(Raw Score) 



Sic:: 25 

CTBS Concepts and Applications versus Covariate (Gates McGinitie) 
Second Grade Class Means; CSMP Classes (x) and Non-CSMP Classes (•) 




Analysis of Class Means 





Site 25 




CSHP 


Non-CSMP 


Number of Classes 


6 


6 


Covariate Mean 


21.5 


20,7 


CTBS Concepts 
(Adjusted Mean) 


18.1 


17.5 


Probability level 
of differences 


.48 



Site 13 

CPT Mathematics versus Covariate (CPT Reading) 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 



CPT 
Mathematics 
(Raw Score) 




Analysis of Class Means 





Site 13 


I 

1 


CSMP 


Non-CSMP 


Number of Classes 


6 


6 


Covariate Mean 


32.8 


35.4 


CPT Mathematics 
(Adjusted Mean) 


36.2 


35.7 


Probability level 
of differences 


.18 



30 



31 



Ti ^ 



34 



IS Zit 37 "^(Percentile Rank) 



Covariate 



i\f)pendix C 



Comprehensive Tests of Basic Skills, lliird Grade, Site 01 

There are two imitli tests in the Conprehensive Tests of Basic Skills: 
Conputation, and Concepts and .Applications. For the purpose of this analysi 
the Conputation Test has been subdivided into four .sections : addition items, 
subtraction iterns, multiplication items and division items. These sets of 
items are also separate sections in the test but are ordinarily combined 
to fora a gross Computation score. 

For each subtest, two kinds of infonnation are given. First, a 
reduced version of each of the various sets of test items are given, 
together with the percent correct for CSMP and for Non-CSMP students. 
(This information is on the left hand page.) An asterisk indicates 
that the difference in percent correct was 10 or more. All such differences 
favored CSMP students. 

Second, a graph of class means is given which corresponds to the 
previous set of test items. CSMP classes are shoun by an x and Non-CSMP 
classes by a dot (•) . (This information is given on the right hand page, 
facing the corresponding set of items.) Also shown on each graph is a 
statistical summary of the class mean data, including the test of signif- 
icance for differences between CSMP and Non-CSMP classes. 

There were 148 CSMP students with a mean score of 50.6 on the 
Kuhlmann- Anders on Test (administered the previous fall) and 148 Non-CSMP 
students with a mean score of 53.6. 

The graphs are presented in the following order: 

addition items 
subtraction items 
multiplication items 

division items . ' 

Conputation Test (Sum of the above sets of items) 
Concepts and Applications Test 
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CTBS Item Analysis, Third Grade, Site 01 



Subtest 1: Addition (12 items) 



Percent Correct 





CSMP 


Non-CSMP 


5 
6 

+4 


99 


98 


6 

0 
+7 


98 


96 


72+73= 


96 


86 ^ 


67 
+18 


85 


79 


79 
+14 


85 


78 


$0.56 
+ 0.68 


72 


64 



Percent Correct 





CSMP 


Non-CSMP 


730 

+593 


73 


70 


32 
10 
+44 


94 


84 ^ 


26+13+14= 


80 


65 ^ 


3,536 
+7 982 


V 1 
/ J 




35 
82 
98 
+ 4 


80 


74 


44 
8,151 
432 
70 

+ 108 


74 


69 



59 



5|u . Site 01 

CTBS Addition versus Covariate (Kuhlmann-Anderson) 
Third Grade Class Means; CSMP Clssses (x) enu Non-CSMP Classes (•) 




CTBS Item Analysis, Tl-iird Grade, Site 01 
Subtest 2: Subtraction (12 items) 



Percent Correct 



• 


CSMP 


Non-CSMP 


9 7-4= 


90 


89 


33 
- 31 


ft? 




88 
- 50 


83 


84 


9 39 

- i£ 


85 

\.<X 


84 


136 
- 82 


75 


84 


149 

- hi _ 


82 


1 " 



Percent Correct 





CSMP 


Non-CSMP 


12S 
- 92 


68 


68 


300 

c 




-JO 


490 
- 130 


82 


87 


498 
- 203 


78 


75 


564 
- 356 


68 


59 


738 
- 169 


60 


51 



9 (Kuhlraann-Anderson) 
) and Non-CSMP Classe 



s (•) 




1 i 1 

of Class Means 





CSMP 


Non-CSMP 


Number of Classes 


7 


8 


- . ^ ■ Mean 
Covariate _^ , , ^ 

I Standard Deviation 


49.8 
7.6 


53.2 
6.8 


CTBS : Mean 

Subtraction , Standard Deviation 

1 Adjusted Mean 


9.3 
.8 

9.5 • 


8.9 
2.0 

8.6 


Probability Level of Differences 


.17 



14- 



j: il 



■l-U. 



u|..'"i 



Covariate 



ERIC 
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CTBS Item Analysis, Third Grade, Site 01 
Subtest 3: Multiplication ("12 items) 



Jx3= 



5x8= 
0 

X 5 

10 
X 6 

32 
X 2 

17 



Percent Correct 
CSMP I Non-CSKP 



87 



89 



82 



88 



82 



67 



96 



96 



89 



91 



76 



66 



25 

X A 

62 
X 3 

53 
X A 

72 
X, 2 

143 
X 2 

113 
X 9 



Percent Correct 



CSMP 
7A 

82 

70 

8A 



76 



63 



Nor. -CSMP 
6A- ^ 
76 
6A 
78 
71. 



63 



ERIC 



nK Site 01 

'^P CTBS Multiplication versus Covariate (Kuhlraann-Anderson) 

Third Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 




CTBS Item Analysis, Third Grade, Site 01 
iii Subtest 4: Division (12 items) 





Percent Correct 






Percent Correct 




CSMP 


Non-CSMP 










375" 


76 


66 




8) 24 


70 


61 


2rs~ 


80 


70 




6) 36 


69 


65 




58 


59 




7)35 


60 


64 




79 


67 




4 ) 124 


60 


46 




67 


63 




24)48 


62 


39 


. 5U0 


70 


62 




90) do 


40 


37 



Site 01 

CTBS Division versus Covariate (Kuhlniann-Anderson) 
Third Grade Class Means: CSMP Classes (x) and Ncn-CSMP Classe;' (•) 




y CTBS, Third Grade, Site 01 

The graph on t}\o next page is for tiie Con^^utation Test whicJi is com- 
posed of tlie four previous sets of 12 items each for addition, subtraction 
multiplication and division. 



CTBS 
Coiputation 
(Raw Score) 



10 



If 



Sitd 01 

CTBS Computation versus Covariate (Kuhlmann Anderson) 
-nird Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (») 



A nalysis of Class Mea ns 



Number of Classes 
Mean 



Covariate 
CTBS: 

Computation ^ Standard Deviation 



Standard Deviation 
Mean 



Adjusted Mean 



CSHP 


Nan-CSI'iP 


7 


8 


49.8 


53.2 


7.6 


6.8 


36.3 


34.2 


4.9 


7.5 


37.2 


33.3 



j Probability Level of Differences 



CTBS Item .\nalysis, Third Grade, Site 01 



Subtest 5: Concepts and Applications (50 items) 

Percent Correct 



1 Whjt don (h«! "3" in 13 sUnd for? 
(J) 3on« iji) IJcni't 

0 3 tons (S^ 13 Ur.3 

2 Which of lh<s*» figutiy u < wjuar©? 




(D 



© 



] LJ 



3 Which of ihifiu rMiiKfak i» tvyenjy- 
thrye? 

Q) 0.23 Gj 2.3 f-:^ 23 (B> 33 

4 Which of lht*\»f (ip'U'." Is 4 t«cUn|£le? 



(0 



V 



V 



Which of th. . 
hundrt?d M X? 

^ 26 
Look aC th 



Which Irfti" \l)Ow< tlnf 
c«»nier of lh/» ;»-r!-'* 



^5 

:'.iii!ffHls is l**o 




-i) E 



G 



Which of nun: 't.il-s -whows ],hr« 

X«ookatthr E 
circl* / ^\ 



.G 



V»'- ..h letter fihu- 
r/jl.it on Ihe circle:* 



O E 0 F 'i^ t^, 'J) H 
Which of lhp«? ji mosl hkt r ctrclt? 
(J) book (5) ch*ir wheel :y5 Hoor 



CSMP 


Mon-CSMP 


46 


42 


86 


88 


84 


84 


76 


71 


'.;V'j 


92 


84 


81 


87 


89 


67 


65 


90 





27 



28 



29 



3D 



31 



NinJ had J Vfl <.f 17 n>.trbl<^. Sh»r 

tosl 6 ln•rblf^k in 4 WhuS nnv 

~' iYmi S.-.IS b«»iow how fr[4ny 
»n*rbl<^ sh« hdd left'' 





Here are a scl of rabbin ^iid a vfl of 
canoU, How many jmou: c^trou are 
there than rabbits'' 





Here a &«t of (!ov^-ts 




Q S S ) 




Whiih one <if thf*^ p-«lr> (if fiytjrrs 
totitd b«» pui ioi;fin«.*i to m«*ke 4( 



Fran WiinU In buy n tirkwl frx 
Which of ihr fon«»vrin« >»howi ho» 
^h<« rould p«v for it'' 
five nivk^-U 

X) * *ro ditnri and ^ u-c)(t*\ 
Of) thr*» .fi . • 



Percent Correct 



CSMP 
82 



82 



58 



69 



85 



CTBS Item Analyr^is, Third Grade, Site 01 
jbtest 5: Concepts and Applications (cowt.) 



Percent Correct 



Percent Correct 



10 Which of lh# !»«ts Mo<« h^i *n »quaJ 
numWr of birds jnd tr»««? 




11 



Which <t( th« Utow « i4rp*i*led 
into fourths? 




12 Miiw nwnv nwnib^r* 4*rt Ihrrr m 



ODO 



(D I C£) 2 0 10 



13 Mr. Smith w<nh«J hi* car. Th« two 
clockt ihow >ou uhtfn he st.trt^d <nd 
wh«n hf finished. At wh<il lima d»^: 
ht finish? 




® 6:40 0 . (i> 8:00 (B) 3'30 

U ^/h»l cenlimrler, .r«l« ; 
inet»r ill mMSur»»? 
0 l»m« CO njnr*y 



(£) weight 



,^ dis'aiicv 



IS 



18 



17 



A dollar hki the wnte valut 

<A) 5 mcktla (C) 10 r.ickeii 

(E) 20 n-cktli ' hicktli 

How nt»ny *rif lh«;r- m one hilf 
do7»n? 

OD 2 0 6 12 .«) 24 

Which duck is '.H>rd in hr,e f-tim iht 
kft"* 



CSMP 




83 


/6 



^7 t-: 



18 



19 



Which of the*** ts ^noil'tr n-mr '-^t 
14^ 



i 19 4 
'I. 6 ^ 7 



'M IC • 3 
N - 7 7 
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R 6 - 6 ♦ n 
« 6 •. « 



78 



Wh:rh or thf fallii-in,! rul.r< iSn 
nunlSrc »»nunrt lcu«? 



c. 12 



0. U 



47 



71 



77 



9 3 



57 



84 



31 



67 



44 



46 



62 



76 



8b 



39 



32 Which of th«>4 show^ how much 
m<inrv .1 Un-doiUr bttt and one cent 

(D $10.01 (D $101 

O olO 10 SllO 

33 V.'hJt i", tS^ ddttf one Meek aft«r 
April 8? 

0) Ar-nl 9 April 14 

O Ai-iil 15 'Sj April 18 

34 Ijny h^d i rope 2 ft»«fl lon^. He cut 
;» !.-If. How long w.tsrrtCh h-U? 

'C L u.chc^ ^0 12 inchfs 
v] / :itrK'5 IB inches 
>i ir - ■ »r - 12 .then a is 

:t (i) 4 6 'o; )2 

3*j ! Wt \\T,v segment belnw is on« inch 
l..r>K<.; ih^n wFment AB? 

r, I-, I^T' |. 1' I' !■ |> 

I 'l- ■ 1 2 3 t ^ 



» 

37 \Sh ■'. i\ I he milking number in 
3, >. •/ ,11 ? 

(?) 1 8 (£19 ® 10 



3S 



30 



40 



41 



42. 



43 



V'l'^!. ^'f>cs in Ihe b<ix t(» make this 
I n"'H. j Si titi'Mce (rue? 

G : V 4 + 8 

© : © 6 (RJ 12 (S) 19 

WSil t tn ihtf box ici make ihli 
numU: «.cnl*jn<e true? 



b < < ' ]0 
(4) 0 ® 



(D 9 



C5) 10 



V - r. y - 4 

If X * y - 2 , then 2 is 



(D 9 



^10 ® 20 



Joe! ^Muj^ht 10 randies. He Jte 2 and 
gjve 3 to P.U. Hnw many candtes did 
he hji»e |«fi? 



(5") 7 



® 8 ® 15 



Mario had 8 p^ir^ (»f virki. Half of 
th»'m were ne*. How m^ny p*ui oC 
.v)r<s were old'* 



® 4 



S 8 



® Ifi 



Sarxh !>dd 2 books .tnd Mxrk h;id 7 
fio<jVi. 'I'hcy gawe 3 bo^iks to J-iihn. 
liuw many botik^ did Sjrah ^nd 
Mjfk hj*e l-ft all toc»':rfr * 



'V 2 



6 



D 9 



12 



liil' i\ LilIiT th^n Ted Ju.in is taller 
th<n Hi'l Whirh of the thr<?e bi»yi l* 



TTed 

V ranrtoi lell 



CSh[P 


Non-CSMP 


74 




66 


35 




38 


36 






73 




66 


'47 




38 


74 




64 


66 




54 


63 




51 


47 




47 


71 




74 


71 




73 


61 




58 


53 




51 



54 

C 



CTBS Item Analysis, Third Grade, Site 01 
Subtest 5: Concepts and Applications (cont.) 



Percent Correct 



20 Which of thes*? \i anolh<r 
nan«« for s<ft . 'on d? 

.X) lit 02nd li) 12ih w2l!nd 

21 Which lhr\€ numbers i* rwiirrsL lo 
500 ori the number hne'' 



(S) 45m C0**9'i 'D ^03 (IT 504- 

22 Which of th*«ve numb*rr$ is iJMier 
lh»n8K.35S'' 

<D 83.355 0 88,558 
8S.B5B 8«.Sfl5 

23 Which of :h» following n:*V« thi» 
numbnt ventence irue'^ 

3 X (2 X 4) - (3 X 2) X □ 
(J) 4 ® 6 ® 8 '^24 

24 Wnich nun-.lHrr pair Movr haj <»dd 

CO(t5.3) (.0{8.2> vi)(24.4) A' (27,3 

25 .'.J .! iiad r>0«. He ^p^nl 1S« fr,r milk 
and found 5*. Whdt ta-i ytm do 
find o*jt how nturh mon^v P.iu! had 
lht.:fi*- 

•5) di>>d<* and ^dd 
(i. s jbU ift *f".d .»dd 
(5 ili^M* -ind multjply 
'ji) st'btTScland multiply 

26 AniLi fit't'd 4 aooirs and Louis 
pickM 9 app'« Whirh opr of the 
i^ls b*?!ow sho*% how TTMr.y -»p;>l« 
they p«rk*d yi> ti)i>*''.K»fr? 



f o-:o \ 



Ml 
1. 



,7 



CSMP 


Non-CSffl' 


88 


88 


59 


61 


61 


56 


50 


32 




46 








54 


Bl 


85 







Percent Correct 





CSMP 


Non-CSMP 


45 


Sandy h^d 24 rvd huttmis. 33 blick 


56 






buttons, 3nd f)6 while buttons. How 




m<tny buttons did she have in *\P 








A 98 1. 103 








c iin . 0 \2:\ 






46 


\J Vis 1 U U nt| w«J *»T« MlvH^, lie Wv^UtU 


D J 


47 




like to have 52 mtj sheJU. How m^ny 




more shells niu>l he find'' 








(T) 23 (i) 29 








(©52 <K> 75 






At 


Ann had 4 ttrcf^rd^ Sh» got 2 more 


57 


68 




recurdi (oi hef sitrthd.iy <ind 1 




record to her sister. Wh^ti cin you do 








to f in d how IT) 4 J ly records she h 4d 








th^n? 








(£) idd ;ind divide 








(¥) ai'-* ,ind subtract 




- 




multiply and dividv 








(S) i»iilti{ity dnd subtract 






48 


Maru had ?0« to buy k(te ^trmK- It 


50 


49 




co5t 10< (rii 60 y.^rdv How many 








yards of stiing could ihe buy'' 








\Xj £11 ysrtis 








<0 COyj.di 








© 120 yards 








(£) 200y*rdt 






49 


Chail«» h^d 7 (xranulsand Vaitn hid 


59 


43 




2 times IS niiiny. flow many p^«i)Uii 








did Pc<lro h^ve'^ 








'ij 9 14 ^c; 16 o; 21 






50 


Mr Alvarez had 36 prizev He divid* 


. 60 


56 




ed the prires equally arr.ma 6 chiU 






dreii, Hiiw in^ny prizn did each 








child get? 








ij6 7"; 8 1< 32 « 42 







7 0 
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Site 01 

CTBS Concepts and Applications versus Covariate (Kuhlmann Anderson) 
Third Grade Class Means: CS}(P Classes (x) and Non-CSMP Classes (») 



CTBS 

Concepts and Applications 
(Raw Score) 



32 




Analysis of Class Means 



ERLQ ^ 



45 



^0 



1' 

• 


CSfO' 


1 — 

N'-n-CS^ff 


Number of Classes 


] 


8 


\ Mean 

^.standard Deviation 


49.8 
7.5 


53.2 
6.8 


CTBS: 


'Mean 


33.5 


31.5 


Concepts and ^ 
Applications 


Standard Deviation 


3.9 


5.9 


(^Adjusted Mean 


34.5 


30.4 


ProbabiUty Level of Differences 


,02 



Covariate 
(Raw Score) 



78 



56 
C 



Coinmcntary 

The difference in mean adjusted scores was about four points in 
favor of the CSMP classes on botli the Coiiputation and the Concepts 
and Applications Tests. These differences were large enough to be 
significant, even with the relatively smll nuni3er of classes involved. 
However, it is interesting to conpare the graphs of these two tests. 
On the Concepts and Applications Test, the data fit the regression 
line quite well and tlie CSMP classes are generally doing better at all 
levels bf ability. Ch the Confutation Test, however, there is much rnore 
dispersion about tlie regression line (in fact for th^e CSMP classes -only, 
X the distribution is more U-shaped than linear); the difference in tlie 
two groi^DS seems to derive from tlie three CSMP classes of lowest ability, 
all of whom have high scores on the Confutation Test (these are tlie x*s 
in tlie upper left portion of the graph). 

In looking at the percentages correct on various items, it can be 
seen that CSMP students seem to do particularly well, relative to Non-CSMP 
students, on items in the Concepts and Applications Test involving open 
sentences and of the foim: ''What number goes in the box to make this 
nunier sentence true? In the set of addition items from the Conputation 

Test, CS^!P students did relatively best on the two items in which the addition 
problem v.as given in. horizontal format; e.g. 72+73= 



7 }) 
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i\ppendix 1) 



CSMP- Specific Tests A and B, Second Grade 

For eacli of CSNP-Specific Tests A and B, similar pages have been 
groi5)ed together in the analysis to form subtests. A total of seven 
subtests were thus fornied and the data from these subtests are pre- 
sented one after the other, in tiie following order: 
CSMP-Test A: Subtest 1, Arrov. Hiagniins 
Subtest 2, 'liniLOinouter 
Subtest 3, integers 
CSMP-Test B: Subtest 1, Arrow Diagrams 
Subtest 2, Minicomputer 
Subtest 3, String Pictures 
Subtest 4, Conputation 

For eacli subtest, three kinds of information are presented: a) item 
analysis data, b) summary subtest data and c) class meaa data. 

a) Item Malysis Data. For eadi test page, a page appears in the 
Appendix on whicli three pieces of information are giveh: 
i) At the top of the page are any directions for tiiat page that 
were given by the tester to 1±ie whole class before they 
began that page, \N4iere such were considered necessary. 
It should be noted that these tests contained no new 
kinds of problems (problems not found in regular workbool3), 
and that additional directions, sometimes given individually 
to students, are not given here, 
ii) In the middle of the page is a reduced version of the actual 
test page. For eacli item on this test page two kinds of 
data are given. A circled entiy ((^)) indicates the percent 
correct based on ail students who took the test (n=650-670) 
and a bar graph entry shows the percent correct for each of 
five ability levels (quintiles) of students. These bar graph 
percentages were based on those students for whom a test 
score in reading or general ability was available (n-- 390-410) . 
For exanple the bar graph below shows about 30% correct for 
those students in the lowest quintile (i.e. students whose 
percentile rank on the ability or reading test was less than 
10) y and about 7S-6 correct for rhosc students in the highest 
quintile (i.e. students whose percentile rank was 80 or more). 



7f 



iii) At the bottom of the page is ^ny miscellaneous data for 
particular test items. 

b) Summary Subtest Data. For the subtest (one or more test pages) 
a frequency distribution and various statistics by quintile 
according to student ability are given. In particular the 
percentage of students in each quintile who got all or nec rly * 
all the items correct is given as is the percent who got rough- 
ly half or fewer of the items correct. Also given are correla- 
tion coefficients between the subtest and various other test 
scores. 

c) Class Mean Data. Hie mean score on the subtest for each class 
which took tlie subtest is plotted on a graph against the mean 
ability or reading score for that class. Through the resulting 
set of points, one for each class, the regression line has been 
drawn. The reader can conpare these graphs with other subtests 
(and tests in otlier appendices) to conpare the degree to which 

. class scores are predicted by the covariate (i.e. the degree 
to which the dots lie close to the regression line). 

Finally at the end of this Appendix, summar>' d?ta, including graphs 
class means, are given for the total of CSMP Test A and of vJSMP Test B. 



81 



59 
D 



■ CSMP Test A, Subtest 1, Arrow Diagrams 

"The problem Is to label each of these dots (point to dots, lower left) with one 
of these numbers (point to numbered dots urithin string, lower right). Remember 
each of the numbers may only be used once. You may have to erase some numbers 
if it doesn't work out right." 




The item statistics are given for each arrow and each was scored independently. 
An item (arrow) was counted as correct if the dots at the ends of the arrow were- 
labelled so as to ''fit" the relation defined by the arrow. For example, the left 
most blue arrow was counted as correct if the ends were (left to right) 4 and 6, ^ 
or 9 and 11, or 12 and lA. Fifty-six percent of the students were able to label 
the 6 dots in the one way that "fit" the arrows, namely 9, 11, 12, lA respectively 
along the top set of arrows and A, 6 for the lower arrow. 



ERLC 
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CS^[P Test A, Subtest 1 Arrow Diagrams (cont'd) 



6£ 
D 




Build a road between 1 and 8, 
Use just these arrows. 



d 



A6 



This page was scored on three dimensions, Independent of one another: 
a) Use of only the given arrows (+3, +2) 



I 



@ 



b) Construction of a road which did end at 8; 



—I 



c) Computational accurrcy of all arrows,. 



ERIC 



Altogether'^ 3 8 /o of the students were 5ble to draw a road from 1 to 8 with the 
given arrows (i.e. were corret: on all 3 dimensions). 

'66 
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CSMP Test A, Subtest 1, Arr o w Diagrams (cont'd) 



Label the dots. 



+ 3 




Each arrow was scored independently and was considered correct if the dots at 
the end of the arrow were labelled so as no *'fit" the relation defined by the 
ary'ow. The heavy arrows, above, were colored red on the test. 

Twenty-one percent of the students omitted the last question ("What could the red 
arrows be for?**) and this number included significant numbers of students in each 
quintile. Wrong answers to this question tended to be quite different; no single 
wrong answer was given by many students. 



8- 
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CSMP Test A, Subtest 1: Arrow Diagrams 
Frequency Distribution; All Students Combined 



Percent of students 
with indicated score 



30 

10 -! 



r-r 



T r 



0 1 2 3 4 5 6 7 8 9 10 11 12 

Summary Statistics by Ability Group 



Score on Subtest 1, 
out of 12 items 



Ability Gioup^ 



Mean: 
Subtest 1 



Standard 
Deviation 



Percent of students 
with score 0-6 



Percent of students 
with score 11 or 12 



Ql 
Q2 
q3 
Q4 
Q5 



5.7 
8.1 
8.3 
10.1 
10.7 



3.6 
3.6 
.3.3 
2.5 
1.9 



55 



50 



[1 



38 
34- 



€5 
72 



All Students Combined 



8.6 



3. A 



□25 



^Ql means the lowest quintile, i.e. this is the set of students whose 
percentile rank, on whatever ability test was used as a covariate, was 
less than 20. Q2 is the set of students whose percentile rank was from 
20 to 39, etc., and Q5, the highest quintile Is for students whose 
percentile rank was 80 or more. 



Correlations Between Subte st 1 and Other Tests 
Correlations with (number of students); 



Kuhlmann-Anderson: 


.67 


(124) 


SAT Reading: 


.50 


(83) 


CPT Reading: 


.33 


(95) 


Gates McGinitie: 


.44 


(75) 


MANS, Test A: 


.66 


(294) 


MANS, Test B: 


.79 


(142) 


CTBS Math: 


.79 


(263) 



Reliability/Homogeneity (KR20)= . 87 (652) 
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CSMP Test A: Arrow Diagrams Versus Covariate 
Second Grade Class Means 



i 



\ 



X 



(0 



Covariate 



' (Percentile Rank) 

87 



CSMP Ti jt A, Subtest 2, Minicomputer 




Fewer than 2% of the Ffudents omitted even one item. On the third item, 29% of 
the students misread one of the three boards (a very common wrong answer 
being "154"), and on the fourth item, 12%_gave a response of either 780 or 78 
(instead of 708) . 



8« 
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CSMP Test A, Subtest 2, Minicomputer (cont'd) 

"It says, *Write the numbers on the Minicomputer using exactly 2 checkers.' 
Remember you have to use exactly 3 checkers for each one, even if it's easier 
to do it with 4 checkers or 2 checkers." 



Write these numbers on the Minicomputer using EXACTLY three 
checkers. 



4 



= 6 



14 
10 



A8 



® 



@ 




1 1 

^3) 



= 24 rfl" 



60 



An average of 9% of the students showed the required- number on the Minicomputer, 
but with either fewer or more than 3 checkers. An average of 14% of the students 
used exactly 3 checkers but did not di splay rlie reqi . ;ed number. 



66 



CSMP Test A, Subtest 2: Minicomputer 
Frequency istribution: All Students Combined 



Percent of students 
with indicated score * 

20 



0123A56789 10 



Score on Subtest 2, 
out of 10 items 



Summary Statistics by Ability Group 



Ability Group 



Ql 
Q2 
Q3 
Q4 

Q5 



Mean: 
Subtest 2 



4.0 
5.8 
6.5 
8.0 
8.2 



Standard 
Deviation 



3.3 
3.2 
2.7 
2.3 
2.3 



Percent of students 
with score 0-4 



67 



34 



2^ 



Percent of students 
with score 9 or 10 



50 



40 
45 



All Students Combined 



r..6 



3.0 



^Ql means the Icwest qu^vcile i.e. this is the set of students whose 
percentile rank, on whatever c^hilicy test was used as a covariate, was 
less than 20. Q2 is th.=^. set of sm.dents whose percentile rank was from 
20 to 39, etc., and Q5 , Lhe highest quintile is for students whose 
percentile rank was 80 ur more. 

Correlations Between Subtest 2 and. Other Tests 



Kuhlmann-Anderson : 


.64 


(124) 


SAT Reading: 


.37 


(83) 


CPT Reading: 


.23 


(95) 


Gates McGinitie: 


.48 


(75) 


MANS, Test A: 


.69 


(294) 


MANS, Test B: 


.76 


(142) 


CTBS Math: 


.77 


(263) 



Reliability/Homogeneity (RR20)= . 86 (652) 



Dm 



nUj CSMP Tesi A: Minicomputer Versus Covariate 

Second Grade Class Means 

CSMP Test' A • . 
Kinicomputer 
(Raw Score) 
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CSMP Test A, Subtest 3, Integers 



Write N , = or < . 



1^ 



I 13 



12^+ 8 



0 



n 



< 



© 



0 



7_ 
0 



n 



15 



1 




Complete 



AS 



In the top set of items: for the two items without "hats" (/N), an average or 89% 
were correct; for the two items with a negative number on only one sir;<^, an average 
of 48% were correct; and for the two items with negative numbers on each side, an 
average of 30% were correct. Alternatively, 25% of the students systematically 
reversed the inequality sign on all A items involving negative numbers and a further 
14% did the same for only the two items with negative numbers on both sides. 

In the bottom set of items: an average of 18% of the students did the -computations 
as if there were no "hats" (gave responses of— 14, 11, 9, 17 respectively) and 13% 
did this systematically on all 4 items; an average of 14% of the students gave the 
correct absolute value but omitted the hats for the second and fourth items (i.e. 
the two items in which the given negative numben was larger in absolute value than 
the given positive nuniber) , while negligible percentages of students did the revers 
added hats where they shouldn't have been - on the firsc and third items. 



e - 
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CSMP Test A, Subtest, *3: Integers 
Frequency Distribution; All Students Combined 



Percent of students 
with indicated score 



20 
10 



0123^56789 10 



S V ;re on Subtest 3 
out of 10 items 



Summary Statistics by Ability Group 



Ability Group ^ 



Mean: 
Subtest 3 



Standard 
Deviation 



Percent of students 
with score 0-5 



Percent of students 
with score 9 or 10 



Ql 
Q2 
Q3 
Q4 
Q5 



3.9 
3.9 
5.0 
5.6 
6.3 



1.9 
2.1 
2.4 
2.4 
2.3 



All Students Combined 



5.0 



2.6 



7| 



75 



4^2 



10 

10 



11 



SI 



^Ql means i'\e lowtist qiiiritile, i.e. this is the set of students whose 
percentile rank, on. whatever ability test was used as a covariate, was 
less than 20. Q2 is the set of students whose percentile rank was from 
20 to 39, etc., and Q5, the highest quintile is for students whose 
percentile rank was 80 or more. ^ 

Correlations Between Subtest 3 and Other Tests 



Correlations with (number of students); 



Kuhlmann-Anderson: 


.39 


(124) 


SAT Reading: 


.34 


(83) 


GPT Reading: 


.17 


(95) 


Gates McGinitie: 


.47 


(75) 


MANS, Test A: 


.57 


(294) 


MANS, Test B: 


.54 


(142) 


CTBS Math: 


.54 


(263) 



Reliability /Homogeneity (KR20)=. 76(652) 



FRir 



H 



CSMP Test A: Integers Versus Covariate 
Second Grade Class Means 



CSMP Test A 
Integers 
(Raw Score) 




CSMP-Specif ic Test A, Intercorrelations (n=680) 



Subtest 1: Arrow Diagrams 
Subtest 2: Minicomputer 
Subtest 3: Integers 



Subtest 1 



Subtest 2 



.68 
.49 



.48 



Correlations Between Total Score, Test A and Other Tests 

Cortelations with (number of students): Kuhlmann-Anderson .69 (124) 

SAT Reading .52 (83) 

CPT Reading ..35 (95) 

Gates McGinitie .53 (75) 

MANS, Test A .77 (294) 

MANS, Test B .86 (142) 



CTBS Math: 



.82 (263) 
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CSVJ^ Test B, Subtest 1, Arrow Diagrams 

"It says 'Find th:. nuii^ber ^3 and circle it,' That means that one of the dots on 
this page is for f You have to figure out which one it is and then circle it. 
You can label any uotii you want to help you out." 



Find the number 83 and circie it. 

+2. 




This page was scored on two criteria: 

a) Computational accuracy; no dots labelled incorrectly according to the +2 
arrows . 




b) Circling (or labelling) the correct dot for 83. 




1 0 0 
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CSMP Test B, Subtest 1> Arrow Diagrams (contM) 




Each arrow was scored independently and wa': considered correct if the dots at the 
end of the arrow were labelled so as to "tit^* the relation defined by the arrow. 

Fifty-two percent of the students labelled all dots correctly. Averaged across 
itemsj only 2% or the students reversed the direction of the arrow and 5% of 
the students used the wrong arrow (i.e. used the +10 relation instead of the +1 
relation, or vice versa). 
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CSMP Test B, Subtest 1, Arrow Diagrams (cont'd) 



Label the dots. 




® 



® 



What could the red arrows be for? 



J 



Again, each arrow was scored independently. The heavy arroirs were originally red. 

The last question, "What could the red ;?rrows be for?", was omitted by 16% 
of the students. The most frequent answers besides "-8" were "+8" and 

"8" (with neither a plus nor a minus sign). 

In labelling the dots, an average of 5% of the responses contained a computational 
error of ±1 and an average of only 2% of the responses used the relation "-HA" 
instead of "-4". 



CSMP Test B, Subtest 1, Arrow Diagrams (cont'd) 
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An average of 4% of the responses per item evidently used the wrong relation 
instead of "2x", or vice versa) and the same percent evidently used the wrong sign 
(XI instead of +1, or +2 instead of 2X). 

Sixty-one percent of the students had all dots correctly labelled. 
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CSMP Test B, Subtest 1: Arrow Diagra:T:s 
Frequency Distribution: All Students C p^ rbined 



Percent of students 
with indicated score 



O.l 2,3 4.5 6.7 



r-rTi 



r 



.9 )0»I1 12.13 U.I5 16. 



Score on S'lbtest 1, 
out of 17 items 



Summary Statistics by Ability Group 



Ability Group^ 



Mean: 
Subtest 1 



Standard 
Deviation 



Percent of students 
with score 0-9 



Percent of students 
with score 16--18 



Ql 
Q2 

Q3 
Q4 
Q5 



8.6 
9.0 
11.6 
14.3 
15.4 



4.8 
5.0 
4.4 
2.9 
2.2 





46 


















3 





(4 



All Students Combined 



12.3 



4.6 



^Ql means the lowest quintile, i.e. this is the set of students whose 
percentile rank, on whatever ability test was used as a covariate, was 
less than 20. Q2 is the set of students whose percentile rank was from 
20 to 39, etc., and Q5, the highest quintile is for students whose 
percentile rank was 80 or more. 

Correlations Between Subtest 1 and Other Tests 



Correlations with (number of students); 



Kuhlmann -Anderson : 


.72 


(153) 


SAT Reading: 


.58 


(97) 


CPT Reading: 


.24 


(98) 


Gates McGinitie: 


.52 


(62) 


MANS, Test A: 


.71 


(143) 


MANS, Test B: 


.69 


(313) 


CTBS Math: 


.78 


(241) 



Reliability /Homogeneity (KR20)=.91(671) 

10 
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I 



To 



Covariate 
(Percentile Rank) 



CSMP Test B, Subtest 2, Minicomputer 



® 



@ 



Write these numbers on the Minicomputer. 



04 




' % hi 



271 




= 805 
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Between 5 and 13 percent of the correct answers to the above items were in what 
might be called "non-standard" form (for example "6" can be shown as a 4 and a 
2, or as six I's, or as a 4 and two I's, etc.) 

An average of 3 percent of the responses were, wrong because of an error on one 
board and another 3 percent involved a digit reversal of some kind. 

Ninety-six percent of the students tried all items; 1 percent omitted all items. 



10'/ 
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CSMP Test B, Subtest 2, Minicomputer (c ontM) 



What number is written on the Minicomputer? 




Ninety-four percent of the students attempted all items; about 4 percent omitted 
all. Incorrect answers were not readily categorizable except that the most 
coiranon incorrect responses for the second question were "8" and "84" (instead 
of "12") and for the third question "5" instead of "10") . 
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CSMP Test B, Subtest 2 :. Minicomputer 
Frequency Distribution; All Students Combined 



Percent 

with indicated score 



» T 

10 i. 



— — r I I I i , ' . v Score on Subtest 2 
12 3 4 5 6 7 8 9 nl ^ out of 10 items 



Summary Statistics by Ability Group 



Ability Group' 



Ql 
Q2 

Q3 
QA 
Q5 



All Students Combined 



Mean: 
Subtest 2 



5.2 
6.0 
7.7 
8.7 
9.3 



7.5 



Standard 
Deviation 



3.7 
3.6 
2.8 
2.0 
1.6 



3.1 



Percent of students 
with score 0-6 



S2 



25 



□2« 



Percent of students 
with score 9,10 



2? 



57 



58 



iQl means the lowest quintile, i.e. this is the set of students whose 
percentile rank, on whatever ability test was used as a covariate, was 
less than 20. Q2 is the set of students whose percentile rank was from 
20 to 39, etc., and Q5, the highest quintile is for students whose 
percentile rank was 80 or more. 

Correlations Between Subtest 2 and Other Tests 



Kuhlmann -Anders on : 


.62 


(153) 


SAT Reading: 


.53 


(97) 


CPT Reading: 


.17 


(98) 


Gates McGinitie: 


.38 


(62) 


MANS, Test A: 


.64 


(143) 


MANS, Test B: 


.58 


(313) 


CTBS Math: 


.65 


(2A1) 



Reliability /Homogeneity (KR20)= . 90 (67 1) 
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CSMP Test B: Minicomputer Versus Covariate 
Second Grade Class Means 



X 



X 



JERIC 



30 



1^ (Percentile Rank) 



Covariate 



4 A <{ . 



8S 
TT 



CSMP Test B, Subtest 3> String Pictures (cont'd) 

"The xred string is for numbers which are less than 5 and the blue string is for 
even numbers like 0, 2, 4, 6, 8, 10 and so on. 'At the bottom it says, 'Draw and 
label a dot for 2.' You have to find where the dot for 2 goes and then write '2' 
beside it. Then do the same for. the other numbers.** * 




Averaging across the four items, 9% had the given number misclassif ied with reg^ird 
to the blue string dimension only (e.g. the dot for 2 was placed correctly within 
the red string but incorrectly outside the blue string), 1A% had the given number 
misclassif ied with regard to the red string dimension only, and less than 5% of the 
students had the given number misclassif ied on both categories. 
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CSMP Test B, Subtest 3: String Pictures 
Frequency Distribution: All Students Combined 



Percent of students 
with indicated score 



20 
10 













1 










1 













0 12 3 



Score on Subtest 3, 

^ out of A -i *-(»mr. 



out of A itemn 
Summary Statistics by Ability Group 



Ability Group^ 



Ql 
Q2 

Q3 
Q4 

Q5 



All Students Combined 



Mean: 
Subtest 3 



2.0 
2.0 
2.2 
3.0 
3.5 



2.6 



Standard 
Deviation 



1.3 
1.3 
l.A 
1,2 
0.9 



1.3 



Percent of students 
with score 0-2 



G5 



J7 



4^ 



Percent of students 
with score 4 



IB 



If 



^Ql means the lowest quintile, i.e. this is the set of students whose 
percentile rank, on whatever ability test was used as a covariate, was 
less than 20. Q2 is the set of students whose percenitle rank was from 

' 20 to 39, etc., and Q5, the highest quintile is for students whose 
percentile rank w^xs 80 or more. 

Correlations Between Subtest 3 and Other Tests 



Correlations with (number of students); 



Kuhlmann -Anders on: 


.51 


(153) 


SAT Reading: 


.54 


(97) 


CPT Reading: 


. 10 


(98) 


Gates McGinitie: 


.56 


(62) 


MANS, Test A: 


.67 


(143) 


MANS, Test B: 


.50 


(313) 


CTBS Math: 


.50 


(241) 



Reliability /Homogeneity (ICR20) = . 63(671) 
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CSMP Test B; String Pictures Versus 
Second Grade Class Means 



Covariate 



So 



^0 



60 



Covariate 



It 



115 



8<) 
17- 



CSMP Test B. Subtest 4. Computation 



Complete 





1^ 



2x37=jxd 



I 

2 

1 
2 



85 
31 





X 48 = 



01 



B7 



There were three incorrect responses given by at least 5 percent of the students. 
Two of them had to do with misreading the operation signs; adding (85+31) instead 
of subtracting and adding (2+6) instead of multiplying. The other had to do with 
carrying, in the second addition problem, where many responses of "51" or "511" 
were given. 
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CSMP Test B, Subtest A; CoTiputation 
Frequency Distribution: All Students Combined 

Percent of students 
with Indicated score 



Summary Scatlstlcs by Ability Group 



Ability Group ^ 


Mean: 
Subtest A 


Standard 
Deviation 


Percent of students 
with score 0-3 


Percent of students 
with score 6,-7 


Ql 
Q?. 
Q3 
QA 

Q5 


3. A 
3. 1 
A. 3 
5.7 
6. 1 


2.3 
2.1 
2.2 
1.4 
1.2 










I'M 


20 


\% 


14. 












All Students Combined 


4.8 . 


2. 1 


1 Ut 


ISO 



Ql means the lowest quintlle, i.e. this is the set of students whose 
percentile rank, on whatever ability test was used as a covarlate, was 
less than 20. Q2 is tha set of students whose percentile rank was from 
20 to. 39, etc., and Q5, the highest quintlle is for students whose 
percentile rank was 80 or more. 



Correlations Between Subtest 4 and Other T ests 

Correlations with (number of students): Kuhlmann-Anderson: .69 (153) 

SAT Reading; .58 (97) 

CPT Reading: .32 (98) 

Gates McGinitie: .47 (62) 

MANS, Test A: .77 (143) 

MAT^S, Test B: .70 (313) 

CTBS Math: .77 (241) 

Reliability/Homogeneity (KR20)=. 80 (671) 
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0 1 2 J 4 5 



Score on Subtest 4, 
^ ^ out of 7 items 



CSMi' Test B: Computation Versus Covariate 
Second Grade Class Mc-ans 



X X 



X X 



(0 



Covariate 



1Q (Percentile Rank) 



119 



CSMP-Speclftc Test B, Intercorrelations (n=699) 

Subtest 1 Subtest 2 Subtest 3 



Subtest 1: Arrow Diagrams 

Subtest 2: Minicomputer 

Subtest 3: String Pictures 

Subtest A: Computation 



.72 
.49 
.77 



.71 



.51 
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Test A 



CSM?-Specific Tests A and B. Correlations (n=lll) 





Test B 




Arrow 




String 








Diagrams 


Minicomputer 


Pictures 


Computation 


Total 


Arrow Diagrams 


.62 


.52 


.38 


.66 


.69 


Minicomputer 


.52 


.56 


.41 


.58 


.63 


Integers 


.30 


.28 


.27 


.35 


.36 


Total 


.63 


.59 


* .46 


.69 


.73 



Correlations^ Between Total Scc^e Test B and Other Tests 
Correlations with (number of students): 



Kuhlmann-Anderson : 


.74 


(153) 


SAT Reading: 


.64 


(97) 


CPT Reading: 


.28 


(98) 


Gates McGinitie: 


.53 


(62) 


MANS, Test A: 


.79 


(143) 


MANS, Test B: 


.74 


(313) 


CTBS Math: 


.82 


(241) 



CSMP Test B; Total Score Versus Covariate 
Second Grade Class Means 




Conunentary 

The figure on the next page sliows a* grapli for each of the seven 
subtests of CSMP-Specific Tests A and B. For each graph, there are 
dots which show the mean percent correct on that subtest for the set 
of students in Ql (with percentile rank on covariate <20) , in Q2, in 
Q3, in Q4 and in Q5. With the exception of subtest B3, tha line 
segments joining the Q4 dots to the Q5 dots have a fairly gentle slope - 
they do not go much higlaer. However, for the line segments joining the 
Q3 dots to the Q4 dots, there is an almost uniformly large increase in 
the percent correct. This means that there is very little difference 
in performance between the two highest ability groips, but between 
those two groups and Q3, the middle* ability group, there is a dramatic 
dif^ence^ and one is struck by the regularity of the graphs in this 
regard. It may be that somewhere around the 60th percentile there is 
some type of break in the distribution; above this point students do 
very well, usually over 80^ correct, while the middle group of students 
are getting between 50 and 70% correct (always excepting subtest A3, 
Integers). 

Below Q3, tlie graphs lose their regularity. The Q2 dots are always 
below the Q3 dots of course, but there is much variation in the size of 
the difference. Between students in the lowest quintile, Ql, and student 
in the next-to-lowest quintile, Q2, there are four- subtests on which 
there is virtually no inprovement in performance; it is not until Q3 that 
one sees an inprovement in performance. 



Mean Percent Correct by Quintile, 
CSMP-Specific Subtests 




^\ppendix E 



CSMP-Specific Individually Administered Test, Second Grade 

This test was administered on a one-to-one basis to students from 
37 CSNP second grade classes in 6 sites. Four students, of varying 
but representative ability levels, were selected from each class. The 
test was administered by special testers, trained by the CSMP coordinator 
at eacli site, and required about 15 to 20 minutes per student. 

The format for administering and recording responses from the test 
was quite lengthy and is not reproduced in full here. Instead a rather 
shortened version of the test items and materials is given, togetlier 
with tlie percent of the students who got each item correct. The testers 
were encouraged to explain the questions as fully as possible, with 
alternative wording where appropriate and, with the first two or three 
of a sequence of similar tasks, to show the student how to do tlie question 
if necessary. 

The percentage correct is based on all 148 students who took tlie test. 
About half these students were from classes from which coVariate scores 
(from reading or general ability tests) were available. For these classes 
it was possible to determine the degree to which the selected students were 
representative of the classes from which they were chosen. The mean percen- 
tile rank of the selected students was between 5 and 10 points higher than 
the mean percentile rank for all students, in these classes. Hence these 
students tend to be slightly higher in ability and, to the extent to which 
one might make corrparisons between data from tliese individually administered 
tests and the grouj) administered CSMP Tests A and B, one should bear tins 
in mind. 

On the last page of this Appendix, after tlic test items and item data 
have been presented, some summary data for each of tlie subtests (Minicomixite 
String Pictures, and Arrow Diagrams) is presented. 
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Subtest 1; Minicomputer 



Note: after each question in sequence below, if the student was unable to do the 
question (i.e. arrive at the configuration shown at the right), the tester showed 
the student how to do it. Hence each new question started with the same configu- 
ration for each student. 



1. "Show me one hundred and thirty-seven on the Minicomputer.'* ^8, 

2. "Now you add 28 to the number on the Minicomput^^r . " (So) 



"Figure out what the Minicomputer says. You might have to make some plays." 

a) Made 8-+'2->10 play ^- 

b) Made 10+10-^^20 play (89) 

c) Made 20+20-^40 play (79) 



5. "Let's make the number twice as big. How would we multiply ^^.^ 
it by two?" (69) 



Note: before each of items 6-11 below was asked, 
the Minicomputer was returned to the con- 
figuration shown at the right. 



d) Correctly read off the number displayed on the Minicomputer (^7, 
"Subtract 41 from that number." (84) 





• • 








• 




• 


•• 



"I'm going to move one of the checkers to a new square. Wlien I've done that, you 
tell me whether I've made the number on the Minicomputer larger, smaller, or 
whether it's still the same." 

6a) "I'm going to take this checker (pick up a checker from the I's square) and 
put it here (put it on the 4's square). Did I make the number larger, or 
smaller, or is it still the same?" (91) 



b) (If correct) "How much bigger did I make the number?" (37) 
7a) Same as 6a), but move a checker from the 8's square to the 2's square. (79) 
b) (If correct) "How much smaller did I make the number?" ^6) 

8. Same as 6a), but take two checkers from the 4's square, put one on the 8's 
square and discard the other one. 

9. Same as 6a) , but move a checker from the 200 's square to the 20's square. ^8) 
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10. Same as 6a), but move a checker from the AO's square to the 400 's square. 

11. Same as 6a, but take a checker from the 20 *s square and put it, plus a 
new checker, onto the 10 's square. 

12. *'Now you move a checker to make the number on the Minicomputer larger. 
Make it larger by 1. Make it shdw a number that's one more than the 
number that it shows now.** 

13. "Move a checker to make the number larger by 40." 

14. "Now make it larger by 7," @ 

15. "Now make it smaller by 3. (Ti 



o 
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Subtest 2: String Pictures . 

"Look at this string picture. (Reduced version below was originally in color.) 
This big string is for children in the class. The red string is for boys. And' 
the blue string is for children who are wearing glasses." 




16. "Now what I want you to do is pretend you are in this class. Put a dot for 
where you would go." (9^ 

17. "What can you iiell me about this dot?" (If student being tested is a boy - 
point to top dot; if a girl - point to left-most cioc^) 

a) (If necessary) "is it for a boy or a girl?" 

b) (If necessary) "Does this child wear glasses?"" 



18. "What can you tell me about this dot?" (Point to right-most dot.) 

a) (If necessary) "is it for a boy or a girl?" (fA^i 

b) (If necessary) "Does this child wear glasses?" ^ 

19. "Put a dot for me (for the tester). Wtiere would I go?" 
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Subtest 3: Arrow Diagrams 



'Now let's look at this arrow diagram. (Reduced version, below, was oringinally 
in color.) If someone weren't very good at calculations they could use this 
diagram to help them. Let's pretend we aren't very good at figuring out number 
problems and see which facts we could know just by looking at the diagram ." 

6U 



4i u 



A +15 



3ll 



->- 



"I'm going to give you some problems and you decide whether or not you can figure 
out the answer from the picture." 

20a) "Could we use this diagram to figure out what 31+15 is?" 
b) (If "yes") "Wliat is the answer to 31+15?" (7 



Note: for the first two or three items of this subtest the tester had the student 
show or showed to the student if necessary, how one would use the diagram 
(which arrows from which numbers, etc.). 

21. "Could we use the diagram to figure out 2x92?" ^66) 



22a) "Could we use the diagram for 61-15?" (51, 
b) (If "yes") "What is the answer?" 6l 



23. "Could we use the diagram to figure out 31-15?" 

2Aa) "Could we use the diagram to figure out h of 92?" (53 
b) (If "yes") "And what is the answer?" @ 

25a) "Could we use the diagram to figure out 31+30?" @ 
b) "(If "yes") "What is the answer?" Q 

26. (If correct answer given to //25) "Suppose I draw an arrow like this: 

What could this arrow be for?" (^\ \j\ 



27. (If incorrect answer given to //25, same as //26, except reverse 
the direction (arrow head) of the new arrow.) (24) 



28. (If incorrect answer given to #24) "Suppose I draw an arrow like this: 
What could this arrow be for?" (25) 
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CSMP Individually Administered Test, Summary Statistics 





Number 
of Items 


Mean 


Standard 
Deviation 


Correlation with: 


Sub tes t 1 


Subtest 2 


Subtest 


1: 


Minicomputer 


20 


12.4 


4.7 






Subtest 


2: 


String Pictures 


6 


4.7 


1.1 


.40 




Subtest 


3: 


Arrow Diagrams 


12 


6.0 


2.6 


. 12 


.08 



Correlations Between Individually Administered Testis and Other Tests 





CSMP Individually Administered Tests 








String 


Arrow 






Other Teats 


Minicomputer 


Pictures 


Diagrams 


Total 




MANS A 


70 


.47 


.31 


.73 




MANS B 


49 


.15 


.31 


.50 


CSMP 


Test A: Arrow Diagrams 


.50 


.30 


.11 


.52 




Minicomputer 


.58 


.27 


.04 


.56 




Integers 


.47 


.23 


.28 


.53 


CSMP 


Test B: Arrow Diagrams 


.62 • 


.28 


.55 


.69 




Minicomputer 


.66 


.23 


.63 


.73 




String Pictures 


.58 


.29 


.41 


.63 




Computation 


.62 


,26 


.50 


.67 
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Commentary 



It is not possible to directly conpare students' performance on 
this test with their perfonnance on the CSMP-Specific group administer- 
ed tests. For tlie most part tJie questions in this individual test 
sinply could not be asked in a paper- and-pencil setting. The questions 
were generally more difficult and the testers helped the students with 
explanations, corrections, encouragement, etc. '(Tiey were also told to 
be lerient in the scoring protocol. 

A total of 6 testers in different sites were used and each was 
trained by the respective local CSMP coord.inator. No doubt there was 
some variation from site to site in the way the tests were administered 
and scored. Given these circujnstances it was not feasible to '*open u^)'* 
the testing process to a deeper probing of student responses. (For 
exanple, by asking to explain his or her response, by backtracking from 
points of difficulty, asking progressively harder questions after success 
with easier ones, etc.) Tliat would have yielded more interesting and 
explanatory information but would liave been far beyond tlie scope of 
this study. 

Rather low correlations were found between the String Pictures 
subtest and the group administered CSMP-Specific Tests. Some very low 
correlations were found between the Arrow Diagrams subtest and the 
subtests from CSMP-Specific Test A (though not Test BI). In particular 
the correlation between Arrow Diagrams (individually administered) ajid 
Miniconputer (group administered Test A) was .04. In other words there 
was virtually no relationship between student perfonnance on these two 
tests. IVhetKer this was a function of the variation across testers in 
the administration and interpretation of the test or of the different 
types of items - the ''doing'* as opposed to the '\vriting" - cannot be 
de.termined. 
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Appendix F 

Item Analysis of Tliird Grade CSMP-Specific Tests A and B 

On the following 16 pages are reproduced, in somewhat reduced form, 
the 8 pages which constituted Test \ and the 8 pages which constituted 
Test B. (The actual page numbers of the test are part of that reduced 
page, for exajiple A3 is page 3 of Test A,) 

Three kinds of additional information are presented: 

a) At the top of each page are given any directions for that page that 
were given by the tester to the whole class before tliey began that 
page. This was not usually considered necessaiy, since there were 
no new types of problems nor directions that were not found in 
various regular CSMP workbooks and worksheets. Additional explan- 
ations were given individually to students throughout the test as 
the need arose.- 

b) In the middle of tlie page, superinposed on the reduced test page, 
a circled nunber is given beside each test item, '^'^cept where 
otherwise noted, this is the percent of students who got the item 
correct. 

c) At the bottom of the page are given any other statistics whidi are 
thought to be of interest for that page, such as: percent of students 
who omitted the item, .correlation coefficients with scores from the 
Kuhlmann-Anderson test (administered previous fall), common incorrect 
responses, conparisons with the performance of second grade students 
(where applicable), etc. 

The data from Test A are based on 76 students from 4 local St. Louis 
area classes, TTie mean score on the Kulilmann Anderson test corresponds 
to a percentile rank of approximately 45. ^fhe data from Test B are based 
on 59 students from 3 classes, and their mean score corresponds to a 
percentile rank of approximately 56. 
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Third Grade, CSMP Test A 
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"Look at the first question at the top of the page. It says 'Game 1', and four 
different spinners are shown. Now listen and I'll tell you about this game. 
You will have a hundred spins in the game. After each spin you will get the 
number of points the spinner is pointing at. You think about which of the four " 
spinners would be likely to give you the most points after a hundred spins. Then 
put an X on the spinner that you would choose. Then do the same thing for the 
other games." 



7a mc 1 



1 





1 


4 





jarnc 3 



1 



1^ 



6 



@ @ @ 



3 



5 



Al 



© 



2 



m 
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In Game 2 and Game 3> it is interesting to note tha,t while the most popular choice 
was the best answer, the next most popular choice was the worst answer. 

The mean correlation Detween the three test items and the Kuhlmann Anderson test 
was . 31 . 
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Third Grade, CSMP Test A (cont'd) 



> 


Write these numbers on. the Minicomputer. 
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The mean percent correct for these 5 items was 84- The same page was given to 
second grade students (CSMP Test B) and the mean percent correct for them was 78. 

Approximately 12 percent of the correct a^^swers were given in what might be 
called "non-standard" form. For example, 6 is usually shown on the Minicomputer 
with a checker on the four and two's plac-., but it can also be shown with 3 
checkers on the two's place, and by various other configurations. 

The mean correlation between the five test items and the Kuhlmann Anderson test 
was .50. ' 
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Third Grade, CSMP Test A (cont'd) ^ 




A3 



For the six items, an average of 13% of the students showed the correct sequence 
of digits, but placed them incorrectly in relation to^ the decimal point or the 
decimal bar (colored green on the original). This was more common in the top 
three items. 

For the six items, an average of 26% of the students lefL out a digit, or reversed 
the digits. This usually involved the digit ''0"; more than half the students had 
this problem with the fourth and fifth items, where the;' most often showed the 
numbers 15.036 and A07.008 respectively. 

The correlation between this set of items and the Kuhlmann-Anderson Test was .34. 




The red arrows in the original text are depicted here by the heavier arrows. 

The item statistics are given for each arrov; and each was scored independently. 
An item (arrow) was counted as correct if the dots at the end of the arrow "fit" 
the relation defined by the arrow. 

The mean percent correct for the 5 items was 83. The same page was also given 
to second grade students (CSMP Test A) and the mean percent correct was 68. 

The correlation between this set of items and the Kuhlmann Anderson test was .42. 

Er|c 1^:5 7 
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Third Grade, CSMP Test A (cont'd) 



Label iU dots 

2x 3x -8 

i 
I 




Different colors were used for different valued arrows; the dotted arrow was originally 
yellow. 

Each arrow was scored independently, as on the previous page. The percent correct . 
for one of the red arrows has not been indicated because that item was treated in a 
special way. Unfortunately an error was made in producing this page for test adminis- 
tration and a student doing the question correctly would have dots which were labelled, 
successively, ''1", ^'2", "6" and then '*2'\ Then he would have to multiply i by 3, a 
process not attended to in the curriculum. Therefore when analyzing responses to this 
particular ar^ow, students were divided into two groups: those who had labelled the 
previous dot 2 or some other negative number (52% of the students); and those who had 
labelled it as a non-negative number (A8%), evidently tnaVing a mistake previously. Of 
the first group, 75% were correctly able to multiply t' v negative number by 3. Of the 
second group, only 52% were able to multiply their noi. -negative number by 3. What is 
surprising is not that more of the first group got it right th^an in the second group 
(they were probably better students) but that such a high percent of this first group 
was able to multiply a negative number. 

The most coiomon erroneous answer for the question about the yellov? arrow wns *'5x". 

Each of the last four parts to this item (including t\^e "yellow arrow*^ question) were 
omitted by an average of 12 percc!nt of the students ♦ 

Th<2 correlation between this set of 6 items and the Kuhlmann Anderson test was .26. 

o 
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Third Grade, CSMP Test A (cont'd) 



Label the dots 

IX -He- -3 




A6 



Each arrow was again scored independently. Across items (arrows), the average 
percent of students who omitted an item (i»e* , at least one end of an arrow 
not labelled) was 13, with the most omits occuring with the top (+5) arrow. 

For each arrow, a check was made on how often students evidently used the wrong 
one of the three given arrows, and on how often the student evidently reversed 
the arrow (for example treating the +5 arrow as --5, or the 2x arrow as %x) . In 
only two instances did more than ,5% of che response^. have one of these errors 
And both were arrow reversals: in the left-mu3t 2x arrow (16%) and in the lowest 
-3 arrow (25%). 



The correlation between this set of items and the Kuhlmann Anderson test was .55 



Third Grade, CSMP Test A (cont'd) 



The first question was done together with the class, as an example. Then "The 
others on this page are the same, except that you must read to see how many 
negative checkers to use and what number you are to make." 



/Negative CUcUvs 0 



AJl Z i\uaVwc cUcW^^-t to +V;s HH-Xty to m*V<, 4,472 



@ 



Al^ J» ht^^^^^^ ckccVcrr 'to lUU HunAvtr "to n,cV< 6 



Ail I 4»t^f.\'.>ie tUcVf* to l^Jc v%M>^lc^ to. »^Ac 44 



h7 



For the four items, the average percent of students who omitted an item was 16. 

Most of the wrong answers were answers in tvjhich the correct number of checkers 
was used but the number displayed on the Minicomputer was not correct. 
Never more than 5 percent of Khe. students used the wrong number of negative 
checkers to show the required number. 

The correlation between this set of A items and the Kuhlmann Anderson test was 
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Third Grade, CSMP Test A (cont'd) 

"You can see that this page is like the last one except that you can use positive 
or negative checkers. Or you can use both in the same problem if you want." 
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Students generally used one of two strategies for solving these problems. First, 
they could "cancel" the given number by pairing a negative checker with each given 
(positive) checker and then merely displaying the required number. (For the first 
item one would put a negative checker on 100 and then positive checkers on 80, 10 
and 1.) Or second, they could figure out the difference between the displayed 
number and the required number and then add the right kind of checkers to make up 
this difference. (For the first item 100 is given and 91 is required - henca one 
needs to subtract 9 - hence one would put negative checkers on 8 and 1 or put a 
negative checker on 10 and a positive checker on 1.) The first strategy was the 
more popular, especially with the third item (which could otherwise have been 
solved by merely putting a negatiVL? checker on 1). 

For each item, an average of 19% of th.^ students omitted the item. 
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Third Grade > CSMP Test B 
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After reading the directions at the top of the page: "Look at the first arrow 
picture - the blue one. One of these four dots is for a positive number that 
is smaller than any of the others. You have to figure out which one it is and 
draw a circle around it. You don't have to label any of the dots, but you can 
if you want to. Then do the same for the other pictures and for the ones on 
the next page." 




In about half the correct responses the students had labelled some of the dots to 
help them; In tlie other half no dots were labelled* 



14.-'. 
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Third Grade, CSMP Test B (cont'd) 





What number is .vrillen on the Minicomputer ? 
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The only relatively common incorrect response was "158" (instead of "258") for 
the third item. 

The correlation between this set of 5 items and the Kuhlmann Anderson test was 

14> 



Third Grade, CSMP Test B (cont'd) 



AAA 1 ckeckf to lV»;i numU'- t«. *naW«, 27*? 

9 



© 



AAA 4 clitcWtvj to iiHw^tv lo vnAe 2,2 22. 



54 



There were no cases where students showed the required number on the Minicomputer 
by adding an incorrect number of checkers, but an average of 20% of the students 
were unable to show the required number though they did use the correct number of 
checkers. 

The correlation between this set of items and the Kuhlmann-Anderson test was .^9. 
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Third Grade, CSMP Test u (cont'd) 



Write > , = or < . 



5 


< 


7 


12: 




0 @ 


6 




6 @ 


t 




'8^ ® 


13 




15 @ 


12^+8 










1 © 



Complete 



8+-^ = 








5 + 4 = 


@ 


t4^+ 3 = 


@ , 


85 
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The average percent correct for the 10 items was^ 60. The same set of items kas 
given to second grade students (CSMP Test A) and the average percent correct was 5C. 

In the upper set of items, 15 percent of the students systematically reversed the 
^inequality sign on all four items in which a negative number was present and 
another 15 percent reversed it for only those two items on which there was a 
negative number on both sides. 

In the lower set of items, 15 percent or the students answered all ^ questions 
as if there were no "hats" present (giving a response of *'14'' to the calculation 
8+6, for example). Twenty-five percent of the students gave a response of "3" 
instead of to "4+7^*, and <i response of "11" instead of "fl" to "14+3". 

My 



Third Grade, CSMP Test B (cont'd) 




66 



The mean percent correct for these 6 items was 73. The same page was given to 
second grade students (CSMP Test B) and the mean percen^ correct was 69. 

Common incorrect responses to the last question, regarding the red arrows ,*were 
"+8", "-K*', '*5*' and 'V, though none were given by more than 5 percent of the 
students. (The heavy arrow was red on the original.) 



The correlation of this set of 6 items with Kuhlmann Mderson test was .49. 
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Third Grade, CSMP Test B (cont'd) 



Label i\\t <joU 

■M- '7 +21 



20 




B7 





About 7 percent of the students evidently used the relations -4 or +21 when working 
with the arrow. These were the only i.istances of using the wrong one of the three 
given a'irrows. 

Other than "+11", the most common answers to the last question, regarding the black 
arrows, were "11" (without the plus sign), "8" and "j^". (The dotted arrows were 
originally in black. ) 

The. correlation of this set of items and the .Kuhlmann-Anderson test was .65. 
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Third Grade, CSMP Test B (cont'd) 
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"The blue string is for even numbers. The yellow string is for numbers which are 
multiples of 3. And the red string is for numbers less than 20; Below the picture 
it says, 'Draw and label dots for these twelve numbers.' You can see the numbers 
at. the bottom of the page. For instance you have to find where the dot for 6 goes and 
put the dot where It belongs in the string picture and label it. Then do the same 
for the other numbers." (The strings were appropriately in color on the original.) 




Draw ani 


attl tlois for 


-(ktse iuclve 




6 ® 








if ® 


17® 


2J © 


23® 


24® 


31® 


33© 


38 0 



II 



For each number, the percent of students who placed a dot for that number in the 
correct region is circled besido the number. That circled percent is underlined 
when more than 10 percent of the students did not respond to the corresponding 
number. (ITie mean percent omits, for all items, was 10.) 

The mean percent correct across items was 42. The hardest numbers to place 
correctly were **23" and "31", which were outside all the given strings and, 
surprisingly, "2A", the only number inside the blue and yellow strings and out- 
side the red string. The easiest number to place was "17", which was, in a 
sense, functionally opposite to "24" (i.e., outside the blue and yellow strings 
and inside the red string). 



The correlation between this set of 12 items and the Kuhlmann Anderson test was 
.52. 



Appendix G 

hmS Tests A and B 

Each of the ,M/\NS tests was coii^iosed of five subtests. Separate 
analyses were performeJ for each subtest, and tlie data from these 



analyses are presented 


in 


the following order: 


MANS 


- Test A: Subtest 


1, 


Sequences 




Subtest 


2, 


Equation Fluency 




Subtest 


3, 


Functions 




Subtest 


4, 


Nunber Line Hstiniation 




Subtest 


5, 


Conputation I 


MANS 


- Test B: Subtest 


1, 


Labelling iNumber Lines 




Subtest 


2, 


Sentences .^\bout 8 




Subtest 


3, 


Word Problems 




Subtest 


4, 


Nuirber Sentence Pictures 




Subtest 


5, 


Conputation II 



For each subtest, three kinds of information are presented: a) item 
analysis^ data, b) summary subtest data and c) class mean data. 

a) Item Analysis Data 

For each actual test page, a page appears in the Appendix with the 
following information: 

i) At tlie top of the page, a shortened version of the actual directions 
is given. Sanple problems were discussed with the class and anple 
demonstrations were given. The tester circulated around the room 
to answer questions'- and to check on the students' understanding of 
the directions. Mthough there was a time limit for each siobtest, 
only one subtest was speeded (Number Line Estimation in which 2h 
kninutes was allowed for the 13 items). The total working time for 
^^^\NS A and MANS B was 22 minutes- Time limits for each subtest 
.were determined following pre-testing research and allowed almost 
all students to conplete the subtests, 
ii) In the middle of tlie page, a reduced version of the actual test 
page is given. Next to each test item, the mean percent correct 
for all second grade CSNP students who took the test Cn=337 for 
Test A, 352 for Test B) is given by a circled entry CC^)) . Follow- 
ing tlus entr}* - is the mean percent correct for all second grade 
Non-CSMP students who took the test (n=324 for Test A and 348 for 
Test B). This entry is boxed r rxxl l . 

b) Summary Subtest Data 

At the bottom of the page, the mean scores on the subtest are given 
for all CSMP students and for all Non-CSMP students. In addi-»-ion, 
for each of these two groips of students, mean scores by a^- . !.'i:y 
groT^) are given. For exanple, a mean score is given for all CSMP 
students in quintile 1, i.e. for students with a mean score on the 
covariate test (reading or general ability) of less than 20. 
Similarly mean scores are given for CSMP Q2, Q3> Q4 and Q5 and for 
the various Non-CSMP quintiles. 'Iho distribution of students across 
quintiles is shown below. It can be seen tJiat there are somewhat 



higher percentages of CSMP students in the upper quintil^s; within 
a given quint lie, havever, the wean covariate scores were nearly 
equal . 



MANS A 1 M/iI^S B 





CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


Quintile 1 


13% 


15% 


10% 


15% 


Quint ile 2 


16% 


16% 


12% 


16% 


Quintile 3 


21 


27% 


26% 


29% 


Quintile A 


26% 


2A% 


29% 


24% 


Quintile 5 


20% 


18% 


23% 


16% 


All Students Combined 


100% 


100% 


100% 

1 


100% 



Also given in tlie summary subtest data (bottom of each item analysis 
page) is the reliability (KR201 for the subtest and correlations with 
various other tests (CSMP A, CSMP B, Kuhlmann -Anderson, and standardized 
math tests) . 

c) Class Mean Data 

The mean score on the subtest, for each class which took the subtest, 
is plotted on a graph against the mean score on the ability or reading 
score for that class. Through the resulting set of points, one for each 
class, the regression line of test score on covariate has been drawn. 
These graphs provide a visual conparison between CSMP and Non-CSMP 
classes, and the reader can also see the degree to which class scores 
are predicted by the covariate. In th.^ comer of each graph, summary 
statistical data from the Analysis of Covariance with blocking on 
sites* is given, including adjusted means and p-values. 



^Blocking on sites removes any systematic variation from site-to-site 
from the unexplained error variance. This model assumes there are 
four main sources contributing to tiie variance between scores: 

a) due to entering ability as measured by the covariate (1 degree 
of freedom) i 

b) due to systematic differences between the four sites (3 degi-ees 
of freedom) 

c) due to the differences in curricula (1 degree of freedom) 

d) due to random unexplained error (22 (MANS A) and 24 (MANS B) degrees 
of freedom) 

The P-test conpares variances c) and d) ; if c) is proportionally much 
larger than d) , it is unlikely that differences in curricula are zero 
and produce only random unexplained error. Then the resulting p-value 
is low and when below .05 one says the differences are ''significant'' - 
unlikely to have been produced by chancy f*lcpe. 



MANS A, Subtest!: Sequences 



1^4 



"Look at the first row of numbers on this page. The numbers in the first row are 
11, 12, 13, then a box, then 15, and then 16. The numbers are in a certain order 
and we have to figure out what the pattern is so that we can put the right number 
in the box." (The students were given time to work on this problem and the answer 
was explained.) "Now when I tell you to start you do the rest of the questions on 
this page but be careful because each pattern is different." 



. 12. 13. 



□. 

10. 8. 6. 



2. 0 



1. 3. 5. 



'. 2. □ 



IG, 



O- 9. 11 0 § 



16. 13. 10. 4. 1 @) ^ 

7. 11. 15. Q 23. 27 @ ^ 



32 @ H 



1. 6. 2. Q 3. 8. 4. 9 

@ 0 
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Means 



Correlations (Subtest 1 v Other Tests) 





CSMP 


:;Qn-CSMP 




CSMP 


Non-CSMP 


All Students Combined 


2.9 


2.4 




Reliability (KR20) 


.80 


.77 


Quintile 1 


-1.9 


1.5 




CSMP- Specific Test A 


.61 




Quintile 2 


1.9 


1.1 




CSMP-Specific Test B 


.53 




Quintile 3 


2.1 


2.0 




Kuhlmann-Anderson 


.63 


.63 


Quintile 4 


3.7 


3.0 




Reading Tests ^ 


.43 


.42 


Quintile 5 


4.2 


3.8 




Math Tests^ 


.61 


.61 



^Mean correlation from three different 
standardized reading (math) tests administe 
at various sites. 
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MANS Test A: Sequences versus Covariate 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 



MANS A: 
Sequences ' 
(Rw Score) . 

A .11!!...' 
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MANS A, Subtest 2: Equation Fluency 

"Look at the numbers and things at the top of the page" (the signs and numbers were 
described to the students). We want to write as many true number sentences as we 
can using only these signs and numbers. You can use each sign or number as many 
times as you want in the same sentence. And you don't have to use them all in the 
same sentence." 

(The three sentences given as examples were explained.) 

"Now when I tell you to start, I want, you to write as many other number sentences 
as you can. Work as fast as you can until I tell you to stop." 

= + - ' X 1 2-3 

3 - 1 a 2 

1X3=3 



1 ♦ 1 ♦ 1 ♦ 1 =2*2 







CSMP 






Non-CSMP 




§ Trut 

Sentences 


# False 
Sentences 


# 

Minus 


True Sentences 
# false Sentences 


# True 
Sentences 


# False 

Sentences 


1 True Sentences 
Minus # False Sentences 


All Students Combined 


3.7 


0.6 




3.1 


3.6 


1.0 


2.6 


Qulntlle 1 


2.2 


1.1 




l.I 


2.4 


1.3 


1.1 


Qulntlle 2 


2.9 


0.6 




2.3 


2.2 


1.2 


1.0 


Qulntlle 3 


3.2 


0.7 




2.5 


3.3 


1.3 


2.0 


Qulntlle U 


4.6 


0.5 




4.1 


4.2 


0.7 


3.5 


Qulntlle 5 


4.8 


0.4 




4.4 


5.7 


0.4 


5.3 
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MANS A, Subtest 2: Equation Fluency (cont.) 

"This problem is just like the last page. I want you to write as many true number 
sentences as you c^n. Uork as fast as you can until I tell you to stop." 



= + X i 1 2 




ERIC 



15... 



MANS Subtest 2: Equation Fluency (cont.) 



Total Score (Both Pages) 







CSMP 




Ncm-»CSXP 




§ Trut 

Sentences 


# False 

Sentences 


1 True Sentences 
Minus # False Sentences 


# True 
Sentences 


1 False 

Sentences 


if True Sentencas 
Minus 9 False Sentancaa 


All St;v)denC8 Combinei 


8.0 


1.4 


6.6 


7.0 


2.2 


A. 8 


Qulntxle 1 


4.8 


2.8 


2.0 


4.5 


2.9 


1.6 


Quintile 2 


6.0 


1.4 


4.6 


4.4 


2.3 


2.1 


QulntUe 3 


6.6 


1.7 


4.9 


6.5 


2.9 


3.6 


Quiatile A 


9.9 


UO 


8.9 


^ 8.5 


1.5 


7.0 


Qulntilc 5 


10.7 


0.8 


9.9 


10.7 


1.1 


9.6 



*For purposes of class mean computation, the total number of true sentences was 
multiplied by one-half and tests of significance were based on these numbers. 
Hindsight indicates this was not the best number to use. A more informative 
score (and one which wouio have been more likely to disclose significant differ- 
ences) would have been '*total number of true sentences minus total number of false 
sentences . 



Correlations (Subtest 2 v Other Tests) 





CSMP 


Non-CSMP 


I 

CSMP-Specific Test A 


.60 




CSMT'^Specifio Test B 


.52 




Kuhlmann-Anderson 


.67 


.67 


Reading Tests^ 


.36 . 


.A3 


Math Tests^ 


.53 


.56 



^Mean correlation from three different 
standardized reading (math) tests ndministered 
at various sites. 
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MANS Test A: Equation Fluency versus Covariate 
Second Grade Class Means; CSMP Classes (x) and Non-C3MP Classes (t) 
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MANS A, Subtest 3: Functions 

"Look at the pictures at the top of the page. There is a picture of a student at 
his desk. Then there is a picture of a teacher at the blackboard. Look at the 
numbers in the boxes below these pictures. There's a 5 and beside it there's a 
6. That means a student gave her a 5 and she did something to it and got 6. The 
next two numbers are 8 and 9. A student gave her 8, she did the same thing to it 
and got 9. Then the next two numbers are 15 and 16. Then there is a 20 and an 
empty box. Can you figure out what could go in the empty box?" (The students 
were given time to work on this problem and the answer was explained.) "The teacher 
is adding one each time; 5+1=6, 8+1=9; 15+1=16, 20+1=21. Now, there are 5 other 
problems on the page like this one. For each one, you have to figure out what it 
was that the teacher was doing to the numbers, and then put the right number in the 
empty box. Be careful because she's doing a different thing for each of the five 
problems.' She won't be adding one to any of the others.*' 




Means 



Correlations (Subtest 3 v Other Tests) 



EKLC 





CSMP 


Non-CSMP 




CSMP 


Non-CSMP 


All Students Combined 


2.0 


1.5 




Reliability (KR20) 


.73 


.66 


Quintile 1 


0.7 


0.7 




CSMP-Specif ic Test A 


.54 




Quintile 2 


l.A 


0.9 




CSMP-Specific Test B 


.57 . 




Quintile 3 


l.A 


I. 1 




Kuhlmann-Anderson 


.66 


.62 


Quintile A 


2.6 


1.9 




Reading Tests ^ 


.41 


.39 


Quintile 5 


3.0 


2.7 


1 Math Tests-' 


.52 


.53 



"^Mean correlation from three different 
standardized reading (math) tests administered 
at various sites. 
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i<S Test A: Functions versus Covariate 
Seconl Grade Class Means: CSMP Classes (x)> and Non-CSMP .Classes (•) 



MANS A: 
Functions 



2.0 



(Raw Score ^ i- 1 


CSHP 


Non-CSMP ■ 




Number of llmts . 


16 


15 


3.0 


Covitriate 


'Mean 


56.1 


54.0 




^Standard Deviation 


13.8 


14.8 




MANS A: 


Mean 


2.0 


1.5 




Functions 


^Standard Deviation 


0.8 


0.9 


IS 


. I 


.Adjusted Mean' 


2.0 


1.6 




Probability Level of Differences 


.09 



l.ui. 




)'(Percentil^|^k) 



MANS A, Subtest A: Number Line Estimation 



G 



"Look at this funny looking number line. On the next two pages there will be a lot 
of number problems. For each one you will have to figure out where the answer would 
go on the number line. It could go here between 0 and 10, heie between 10 and 50/' 
(etc. for successive intervals). "You just put a big x on the number line to show 
which two nnmbers the answer goes between. Now there are a whole lot of problems 
and you won' have much time to do them. You should not try to calculate the 
exact answer; Just decide quickly where ' the answer would probably go on the number 
line and put a big x on the right place on the number line." 

(The students were given time to do the sample problem (39+38) on a separate sheet 
of paper, and the answer was explained. The fact that they had only 10 seconds to 
do each problem was demonstrated.) 

"Now open your books to this page. On the next two pages you will have a whole lot 
to do the same way. I'll give you two and a half minutes -.o do them so you'll have 
to work fast. I'll let you know when half the time is up so you can see how you're 
doing. " 



39 + 38 
<-! 



100 + 100 



4- 



-I— * 



@ 0 



325 + 325 
<-i 1- 



4-> 



71 - S9 
<-l 



© 0 



■4-> 



51 + 53 
H 



® 0 



2i» + 29 
^■i \- 



10 M 



® El 



<-< !- 



4-> 



MANS A, Subtest 4: Niomber Line Estimation (cont.) 
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900 - 500 



-I- 



23 + 19 
<-4 



105" - 8 
<-\ 



270 + 270 



♦-4- 



4- 



•5 X 11 
«-J 



100 - 55 



■4- 



T X 16 
^ 1- 



0 E£ 



'@ [1 



4-* 



■4-> 



0 m' 



4* 



Means 





CSMP 


Non-CSMP 






r 

All Students Combined 


4.5 


4.0 


• * 
Qulntile 1 


3. 1 


2.8 


Quintile 2 


3.2 


3. 1 


Qulntile 3 


3.6 


3.5 


Quintile 4 


5.6 


4.2 


Qulntile 5 


6.1 


6.0 



ERIC 



Correlations (Subtest 4 v Other Tests) 





CSMP 


Non-CSMP 


Reliability (KR20) 


.75 


.60 


CSMP-Specific Test A 


.54 




CfflP-Specitif Test B 


.50 




Kuhlmann-Anderson 


.65 


.60 


Reading Tests ^ 


.28 


.34 


Math Tests^ 


.61 


.61 



^Mean correlations from three different 
standardized reading (math) tests administered 
at various sites • 



"1 . 



MAKS Test A: Number Line Estimation versus Covariate 
Second Grade Class Means: CSMP Classes (x^and Non-CSMP Classes (•) 



HUNS A: 
Number Line Estiination 
(lor Scote) 



Nunber of Classes 



Covariate 

HANS A: 
Number Line 
Escinatlon 



Probability Level of Differences 




^-^(PerceatllftRiak) 

7o - 
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MANS A, Subtest 5: Computation I 
"You have to. figure out what goes in the boxes to make the number sentences true." 



5^7. □ 

® 0 


lo-D-? . 


mix 5-15 „ 

® M 


. 2 X 400 -1 1 


CZ] --200 - 100 


3 X 1 1- 300 

® EH 


D-49-0 


ixio-D 

©Hi 


2 y is-D 


600 - 100 -EZ] 
® [il 


a xCH- 2 

0 ES 


35 +1 1- 65 



I 



Means 



Correlations (Subtest 5 v Other Tests) 





CSMP 


Non-CSMP 






All Students Combined 


7.0 


5.3 


Quintile T 


3.3 


3.5 


Quintile 2 


5.5 


3.5 


Quintile 3 


5.9 


A. A 


Quintile 4 


8.8. 


6.3 


Quintile 5 


9.7 


8.5 



99 


CSMP 


Non-CSMP 


Reliability (KR20) 


.86 


.8A 


CSMP-Specific. Test A 


.73 




CSMP-Specific Test B 


.71 




Kuhlmann-Anderson 


.68 


.71 


Reading Tests ^ 


.59 


.47 


Mean Tests ^ 


.73 


.69 



Standardized reading (math) tests administen 

tt. .various sites. 
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MANS Test A, In tercorrelations 
(First Entry: CSMP, Second Entry: Non-CSMP)^ 





Subtest 1 


Subtest 2 


Subtest 3 


Subtest 4 


Subtest 


1: 


Sequences 












Subtest 


2: 


Equation Fluency 


.54 


. 53 








Subtest 


3: 


Functions 


.58 


.54 


.52 .49 






Subtest 


A: 


Number Line Estimation 


.48 


.44 


.53 .45 


.44 .40 




Subtest 


5: 


Computation I 


.6: 


.64 


.64 .65 


.63 .56 


.56 .57 



^Based on 426 CSMP students and 390 Non-CSMP students. 



Correlations Between Total Score» MANS A and Other Tests 





CSMP Students^ 


Non-CSMP 


Students^ 


CSMP-Specif ic Test A 


.77 


(294) 






CSMP-Sp-t : : r^ct B 


.73 


(143) 






Kuhlmanv.^' Aj lu-:": rson 


.78 


(129) 


.79 


(114) 


Gate. K .lOlni r : ; Reading 


.62 


(77) 


.52 


(69) 


SAT H -auin:^ 


.60 


(83) 


.(;3 


(90) 


OPT K..>}?d\-^. 


.47 


(49) 


.53 


(50) 


CTBS 


. 34 


(235) 


.84 


(211) 


SAT Math 


i 


(83) 


. 75 


(90) 


CPT Math 


1 ■■■■5 


(49) 


1 .68 


(50) 



1 



Number in parenthesis is number students on whom correlation is based. 



0\ 



MANS Test A: Total Score versus Covariate 
5?icond Grade Class Means: CSMP Classes (x) and Non-CSMP Classes 



140 
G 



MA^IS B, Subtest 1; Labelling Number Lines 

''Look at the top row of numbers on this page. The numbers on that top number line 
are 1, 4, 7, then a bo:c, then 13, 16, 19 and 22. You have to figure out what number 
would go in the box." (The students were given time to work on this problem and the 
answer was explained.) "'Ten' is the right answer because the numbers are going up 
by three each time; each mark is three more than the one before it. When I tell you 
to start you do the rest of tl ^ questions on this page." 



-I 1 \ 1 1 1 1 1- 

» * ' j ^ j " ^' " 



-I 1 1 1 1 1 1- 



7 9 11 T IJ 



1» U 21 



®0 



-4 1 U 



H i 1- 



.-I 1 



Ut 1«0 I'O 



t*i 115 lit 117 1»« T 



-I 1- 



(^©□©13 




-» 1 1- 



200 220 240 



@I3 



Means 



Correlations (Subtest 1 v Other Tests) 





CSMP 


Non- 


-CSMP 




CSMP 


Non-CSMP 


All Students Combined 


5.1 


4". 


"7 




Reliability (KR2C) 


.85 


.79 


Quintlle 1 


3. 1 


2. 


6 




CSMP-Specific Test A 


.73 




Quintile 2 


3. 3 


3. 


9 




CSMP-Specific Test B 


.46 




Quintile 3 


i.4 


4. 


1 




Kuhlmann -Anderson 


.63 


.56 


Quintile 4 


5.9 


5. 


5 




Rr-ading Tests^ 


.48 


.42 


Quintile 5 


6.9 


6. 


3 




Math Tests ^ 


.68 


.63 



^lean correlation from three different 
standardized reading (math) tests administei 
at various sites. 



H , 
^0 



MANS Test B: Labeling Number Lines versus Covariate 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 



MANS B: 
^beling Number Lines 
(Raw Score) 




1 



ERIC" 




X 





CS.MP 


Non-CSMP 


Nuniber of Classes 


1? 


16 




56.7 


53.3 


standard Deviation 


U.3 


13.9 


,HANS B: 


^Mean 


5.0 


U 


N'uinber line ' 


. Standard Deviation 


' 1.5 


1.4 


Lebelling 


^Adjusted He'an 


5.0 


5.1 




Probability Level of Differences 


.88 



30 



50 



70 



(Percentile Rank) 

m 



MANS B, Subtest 2: Sentences About 8 
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"Look at the top of this page. It says: 'Number sentences • Mary's number sentences 
about 9^ 9=7+2, 9=10-1, 9=1+5+3.' Mary was showing true number sentences about the 
number 9; she was showing things that equal 9.*' (Each of the examples were explained 
to the students-) ''Now on the bottom half of the page it says 'My number sentences 
about When 1 tell you to start, you are to write number sentences for 8. Work 

as fast as you can and write as many as you can think of." 



NuT^bgr Sentences 

Mary's number sentences about 9. 

9-7 + 2 
9-10-1 
9 - 1 5 ♦ 3 



My number sentences about 8, 



8--- 

8 

8 ._ 

8 

8 

8 

8 

8 •_ 

8 



CSMP 



All Students Combined 
Quincile I 
Quintile 2 
Quintlle 3 
Quintile U 
Quintile 5 



True 
Sentences 

8.5 

5.2 

5.^ 

7.6 
10.7 
12.2 



I 



if FalMe 
Sentences 

0.9 

2.2 

0.8 

1.0 

0.7 

0.7 



True Sentences 
Minus 9 False Sentences 

7.6 

3.0 

6.6 
10.0 
11.5 



Non-CSMP 



P True 
Sentences 

6.7 

5.7 
5.9 
7.8 
9.5 



# False 
Sentences 

1.4 

2.0 

2.0 

1.2 

Kl 

0.9 



I True Sentences 
Minus § False Sentences 

5.3 

2.4 

3.7 

4.7 

6.7 

8.6 



'*The graph, next page, used '$x number of true sentences as raw score, 
Correlations (Subtest 2 v Other Tests) 





CSMP 


Non-CSMP 


Reliability (KR20) 






CSMP-Specif ic Test A 


.66 




CSMP-Speclfic Test B 


.35 




Kuhlmaiin- Anderson 


.b5 


.58 


Reading Tests^ 


.34 


.34 


Math Tests^ 


.43 


.45 



ERIC 



^Mean correlation from three different standardized reading 
(math) tests administered at variouiL sites. 



iiL sate 

1 / - 



HANS Test B: Sentences About "8" versus Covariate 
Second Grade Class Means; CSMP Classes (x) and Non-CSMP Classes (#) 



MAKS B: 
Sentences about "8" 
{Ui Score) 

.4 



S 



I ERIC 



30 



1 : 



I ..;.i4; . 




M i I ■ , • 





CSMP 


Non-CSKP 


Number of Classes ' ' 


17 


16 


Covariate 


Mean 

Standard Deviation 


56.7 
14.3 


53.3 
. 13.9 


MAHS B: 




3.9 


3.2 . 


Sentences 


Standard Deviation 


1,3 


1.2 


About 8 1 










Adjusted Kean 


3.7 


3.4 


Probability Level of Differences 





(0 



— ^ (Percentile Rmk) 
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MANS B, Subtest 3: Word Problems 

"There are 8 series of pictures on these two pages. For each of three pictures there 
is a story. At thr end of each story there is a question you will be expected to 
answer. The quest^^ons are written directly below each series of pictures. I will 
read each story aloud two times. Let's all do the first one together." (The 
students were 'given time to work on this problem and the answer was explained.) 

1. "Listen while I tell a story that goes along 
with the pictures of the piggybank. Look at 
the first picture. The story goes , 'At first , 
Mary had some money in her piggybank. * Now 
look at the second picture. The story contin- 
ues, 'Mary's father, added three cents to her 
piggybank.' Now look at the third picture. 
The story goes on, 'Mary broke her piggybank 
and found seven cents.' Below the pictures 
it says, 'How much did Mary have at first?' 
and theii there is a blank. I'll tell the 
story again. At first, Mary had some money 
in her piggybank. Then Mary's father added 
3c to her piggybank. Mary broke lier piggybank 
and found 7C. How much did Mary have at firsc?" 




"First picture: 'David had 11 marbles in his 
marble bag. ' 

Second picture: 'He lost 6 marbles.* 
Third picture: 'David is mad at himself. ' 
Question: 'How many marbles does David have 
now?" 




2. n rtyfcltt 




AT do«« D«vld hav« now? 



"First picture: 'Four children each get the 

same allowance from their mother. ' 

Second picture: 'The four children put 

their allowances together. ' 

Third picture: 'They have altogether 12 

dollars . ' 

Question: 'How much did each child get?' 

"First picture: 'One apple costs 5C« * 
v'econd picture: ^One banana costs 2c. * 
Third picture: 'Sally buys three apples 
and one banana. ' 

Question: 'How much would 3 apples and 1 
banana cost?' 

"First picture: 'The temperature at sunrise 
was 2 degrees below zero.' 

Second picture: 'The temperarure after break- 
fast was .6 degrees above zero. ' 
Third picture: 'Amy wonders how much the temp- 
erature went up between sunrise and after 
breakfast. ' 

Question: 'How many degrees did the temperature 
go up?' 










12 

@ 0 


(J 
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MANS B, Subtest 3: Word Problems (cont.) 



"First picture: fantastic ant is about to 
start a trip. ' 

Second picture: 'Every day it walks 2 miles.' 
'Third picture: 'After 5 days it's almost out 
of sight . ' 

Question: 'How many miles would it walk in 
5 days." 




-OS 



"First picture: 'Mr. Rich lost 100 dollars 
from his wallet. ' 

Second picture; 'Afterwards he still had 
200 dollars. ' 

Third: 'Mr. Rich wonders how much he had 
before he lost the 100 dollars.' 
Question: 'How much did he have to begin 
with?". 



7. 





VcM Much did ht have to btgln with? 




8. "First picture: 'Donald had AO marbles before . 
the game . ' 

Second picture: 'After the game he had only 20 
marbles. ' 

Third picture: 'Donald wonders what fraction of 
his marbles he lost in the game: ^? 1-? 
Question: 'What part of his total did he lose? 
Did he lose .-i or i- or -^? You circle the one 
you think is correct." 





20 Harbl^i 



part of his total dm Uf 




@ 0 
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>\AliS B, Subtest 3, V.ord Problems (cont.) 



Means 





GSMP 


Noii-CSMP 


All Students Combined 


A. A 


3.8 


Quintile 1 


3. 1 


2.8 


Quintile 2 


3.5 


3. 3 


Quintile 3 


A.O 


3. 7 


Quintile 4 


4.8 


4.2 • 


Quintila 5 


5. A 


5.2 



Correlations (Subtest 3 v Other Tests) 





CSMP 


Non-CSMP 






Reliability (KR20) 


.52 


.54 


CSMP-Specific Test A 


.68 




CSMP-Specific Test B 


.22 




Kuh Imanii -An'Ste rson 


.67 


.53 


Reading .Tes'ts^ 


.37 


- .35 


Math Tests ^ 


.45 


.53 



^Mean correlation from'' three different 
standardized reading (math) tests administ' 
in various sites. 



MANS Test B: Word Probleif:* versus Covariate 
Second Lidde Class Means: CSMP Clm^ (x) and Non-CSMP Classes (#) 
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MANS B, Subtest 4: Number Sentence Pictures 

"On this page there are four number sentences. The first one is 12-3=9, and a line 
has been dravn from chat number sentence to one of the dot pictures on the other 
side of the page. That is because that dot picture is the best picture on this 
page tor showing that 12-3=9. When I tell you to start, you have to do the same 
for the next three number sentences. Now after you have done that, there will be 
one picture left over. You are to make up a number sentence that goes with the 
picture that's left over, the one chat didn't match with any number sentence. Just 
write your number sentence here on the blank at the bottom of the page. When you 
have finished this page just go on and do the next page." 




\4eans Correlations (Subtest 4 v Oth^r Tests) 





CSMP 


Non-CSMP 




CSMP 


Non-CSMP 


All Students Combined 








Reliability (KR20) 


.79..... 


.70 


Quintile 1 


4.4 


3. '5 




CSMP-Specific Test A 


.69 ■ 




Quintile, 2 


4.2 


3. 3 




CSMP-Specific Test T 


...33 . 




Quintile 3 


4.7 


3.9 




Kuhlmann-Anderson 


.53 


.52 


Quintile 4 


6.3 


4.7 




/^ading Tests ^ 


.39 


.32 


Quintile 5 


7.3 


6.0 




Math Tests ^ 


.55 


.50 



^Mean correlation from three different 
standardized reading (math) tests administei 
at various sites. 



. ''ANS Test B: Number Sentence Pictures versus Covariate 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 



MANS B: i 
ivnt^x Sentence Pictures 
(Raw Score) 




^(Percentile Rank) 

m 
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MANS B, Subtest 5: Computation II 
'*You have to figure out what goes in the boxes to make the number sentences true." 



® 0 



7X2 



©0 

CUx 2-8 

® 0 

ixn.2 

® a 

300 -LH- 250 



450 



® 0 



10 xQ- 90 
® 0 

423 - 422 

g 13 



50 + 1 50- - 
9 -□- 6 

^ (2 




+ 550 -r~] □-0- 



Means 





CSMP 


Non-CSMP 


All Students Combined 


7.1 


4.9 


Quintile 1 


4.3 


3. 1 


Quintile 2 


4.7 


3.2 ' 


Quintile 3 


5.9 


4.5 


Quintile k 


8.3 


5.7 


Quintile 5 


9.4 


7.7 



Correlations (Subtest 5 v Other Tests) 





CSMP 


Non-CSMP 


^liability (Ki120) 


.85 


.78 


LbMP-Specif ic Test A 


.82 




CSMP-Specific Test B 


.53 




Kuhlmann-Anderson 


.74 


.71 


Reading Tests ^ ^ 


.39 


.49 


Math Tests 1 


.67 


.71 

1 



^Mean correlation from three different 
standardized reading (math) tests administ< 
at various sites. 



MANS Test B: Computation II versus Covariate 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (•) 




M/\NS Test B, Intercorrelations 
(First Entry: CSMP, Second Entry: Non-CSMP)^ 





Subtest 1 


Subtest 2 


Subtest 3 


Subtest 4 


Subtest 1: Labelling Number Lines 
Subtest 2: Sentences About 8 
Subtest 3: Word Problems 
Subtest 4: Number Sentence Pictures 
Subtest 5: Computation II 


.54 .37 
.60 .46 
.52 .36 

.70 .60 

1 


.47 .34 
.44 .37 
.56 .44 


.50 .40 
.67 .56 


.56 .55 



Based on 426 CSMP students and 390 Non-CSMP students. 



Correlations Between Total Score, MANS B and Other Tests 





CSMP Students^ 


Non-CSMP 


Students^ 


Correlation with: CSMP-Specif ic Test A 


.86 


(142) 






CSMP-Specific Test B 


.50 


(313) 






Kuhlmann-Ar.derson 


.79 


(151) 


.74 


(141) 


Gates McGinitie Reading 


.53 


(61) 


.51 


(63) 


SAT Reading 


.71 


(97) 


.49 


(87) 


CPT Reading 


.27 


(47) 


.58 


(56) 


CTBS Math 


.80 


(250) 


. 80 


(224) 


SAT Math 


.80 


(97) 


.78 


(87) 


CPT Math 


.60 

1 


(47) 


.76 


(56) 


^Number in parenthesis is number of students 


on whom correlation 


is based. 



MANS Test B: Total Score versus Covariate 
Second Grade Class Means: CSMP Classes (x) and Non-CSMP Classes (♦) 



' 1 



t I ' ' 



; . 'T • r I ' ' 



Analysis of Class Means 







CSMP 


Non-CSHP 


1 , , J 

1 ■ i ' ! 

t ■ 


Nuiiiber of Classes 


17 


16 




Covariate 


'Mean 

Standard Deviation 


56.7 


53.3 


■i': 

!!'''■ 
■ ' ' ' i ' 


MANSB: / 
Total ^ 


^Hean 

Standard Deviation 


25.8 
6.3 


21.2 
5.5 






^Adjusted Mean 


25.5 


22.5 




Probability Level of Differences 


.02 



so 



to 



-i—)- (Percentile Raak; 

79. wm-: 
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Commentary 

Since the two Conputations Tests produced significant differences, 
items from those tests were categorized according to the kind of confu- 
tation required and are presented below, along with the mean percent 
correct across categories for CSNIP students (first entry) and Non-CSMP 
students (second entry) • 



Pure Addition 

5+7=D 
50+1 50= □ 
450+550=n 

Percent Correct 63,43 



Pure Subtraction 
423-422=Q 

600-100= □ 

9- (_=6 

10- 1^=7 
300-fJ=250 

Percent Correct 60,54 



Pure Multiplication 
7x2=0 
2xl9=G 
2x400=G 
i.xlO=Ll 

Percent Correct 61,27 



Implied Addition- 

C-0=28 
L-2=3 
G-49=0 
L--200=100 

Percent Correct 64,64 



Implied Subtraction 

6+ 0=8 
35+n=65 

Percent Correct 66,67 



Implied Division 
2x0=2 
10x^=90 
3x0=300 

□ x2=8 

□ x5=15 
ixf>2 ^ 



Percent Correct 54,31 



It can be seen that, although TSMP students scored as liigh or 
higher than Non-CSMP students in all categories, the tivo categories 
involving multiplication produced the largest differences. Across all 
items the mean percent correct was 61 (CSNIP) and 45 (fion-CSMP) , For 
the 10 items contaLiing a sign, the percentages were 57 and 29; 

for the remaining items which did not contain a '*x** sign the percentages 
were 64 and 56, 

Alternatively, if one looks at the size of the nujifoers involved, 
another difference becomes apr^irent. There were eight items in which 
at least one of the given numo(urs exceeded 100, 'Ihe percentages correct 
for these items were 51 and 33; for the remaining 16 items the percen- 
tages were 66 and 55, . 

Besides the items in the Computation Tests, there were three items 
in the MANS tests on which the difference in percent correct in favor CSM 
students exceeded 20- These.dealt with \xl6 (Test A4,) , yof 12 (B4) and 
i of a bag of marbles (B3), Also, the two items on the Confutation Tests 
\^ich dealt with fractions had percentages correct of 66 versus 13 and 
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28 versus 8 in favor of CSMP* 

Thus, three areas stand out in which CS^P students did far 
better than Non-CSMP students: fractions (or at least calculating a 
fractional part of), multiplication and working with nunbers larger 
than 100. Tliese areas reflect the larger-than-usual enphasis which is 
apparent in the CSNP curriculum materials. There were of course many 
other test items which produced fairly large differences, but such 
differences were not so dramatic. 



